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In order to create a Statewide evaluation system for 
schools offering occupational education, behavioral objectives of 
various occupational programs were developed A total systems approach 
to education was thus possible with behavioral objectives as the 
basic component. Thirty-six participating secondary schools and 
community colleges in Massachusetts and New York generated over ten 
thousand raw objectives — RAWOBs — during the 1970-72 project. These 
were consolidated into 724 synthesized objectives — SYNOBs — designed 
to provide a means by which schools could select objectives on which 
their students would be tested and by which the schools and teachers 
could follow their own preferences in the processes of instruction. 
Next, a bank of teL'>t items was constructed reflecting the common 
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documents used in the project. A glossary and bibliography are 
appended. (MS) 
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LOCAL EDUCATIONAL AGENCIES (LEAs) SERVED BY ESCOE 



Secondary 
Post-secondary 

Totals 



MASSACHUSETTS 
1970-71 1971-72 
9 9 

3 2 
12 11 



NEW YORK 
1970-71 1971-72 
7 ^ 

3 11 
10 15 



Over 10,000 Behavioral Objectives are in the ESCOE data bank as of 
June 1972. 

Pages viii through xi list the LEA's and LEA personnel that participated 
in some way in the ESCOE project. 
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MASSACHUSETTS 



BLUE HILLS REGIONAL VOCATIONAL 

TECHNICAL HIGH SCHOOL 

100 Randolph Street 

Canton, Massachusetts, 02021 
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Sue Endee 
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Patrick Nicholas 
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Admin: Douglas Til ley 
Facil: Henry Corcoran 

Carleton Kennerson 

Clifford Perry 
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3 Essex Street 
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Stanley Cummings 
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ADIRONDACK COMMUNITY COLLEGE 
Glen Falls, New York, 12801 
Tel. 518: 793-4491 
1971-72 

Admin: Emerson Hibbard 
Facil: C. Elliott Dunn 

Paul Gallipeo 

Bror Wahlquist 

Ronald Williams 

AGRICULTURAL AND TECHNICAL 
COLLEGE AT DELHI 
Delhi, New York, 13753 
Tel. 607: 746-4111 
1971-72 

Admin: B. Klare Sommers 
Faci 1 : Ronald Brach 
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Bronx, New York, 10468 
Tel. 212: 960-8793 
1971-72 

Admin: Richard Donovan 
Facil: Violet Katz 
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Buffalo, New York, 14202 
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1970- 71 

Admin: Joseph Schmidle 
Faci 1 : James Peck 
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Corning, New York, 14830 
Tel. 607: 962-9243 

1971- 72 

Admin: Robert Frederick, Jr. 

Robert Chapman 
Facil: Emory Bauer 

Florence Mealy 

Raymond Welch 



CUNY-ROC 

460 West 42nd Street 

New York, New York, 10038 

Tel. 212: 868-3000 

1970- 71, 1971-72 
Admin: George Quarles 
Facil : Claudia Loftis 

Alfred Dennis 
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Pendal 1 Road 

Poughkeepsie, New York, 12601 
Tel. 914: 471-4500 
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Admin: Lawrence Monaco 
Facil: John Demenkoff 
Robert Dolansky 
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99 Aero Drive 

Buffalo, New York, 14225 

Tel. 716: 634-6800 

1970-71 

Admin: Donald Dayer 
Facil : Donald Dayer 

HUDSON VALLEY COMMUNITY COLLEGE 

80 Vandenberg Avenue 

Troy, New York, 12180 

Tel. 518: 283-1100 

1970-71, 1971-72 

Admin: James Fitzgibbons 

Reuben Merchant 
Facil : Wal ter Long 

Eugene Stalica 

KINGSBOROUGH COMMUNITY COLLEGE 
Oriental Avenue 
Brooklyn, New York, 11211 
Tel. 212: 769-9200 
1970-71 

Admin: Isabel! e Krey 
Facil : Isabel le Krey 
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LA GUARDIA COMMUNITY COLLEGE 
31-10 Thompson Avenue 
Long Island, New York, 11101 
Tel. 212: 937-9200 
1971-72 

Admin: Harry Heinaman 

Freeman Sleeper 
Facil: Dorrie Williams 

NASSAU COUNTY BOCES 
125 Jericho Turnpike 
Jericho, New York, 11753 
Tel: 516: 997-8700 
1970-71, 1971-72 
Admin: Frank Woolf 
Facil : Richard Kresse 

William Steinberg 

Barbara Field 

Jerry Havlik 

Bertram Wallace 

NEW YORK CITY COMMUNITY COLLEGE 

300 Jay Street 

Brooklyn, New York, 11201 

Tel. 212: 643-5015 

1970-71 , 1971-72 

Admin: Mario Iraggi 

Stanley Brodsky 
Facil : Sid Avner 

Carl High 

NORTH HIGH SCHOOL 
East Frederick Street 
Binghampton, New York, 13901 
Tel. 607: 726-2466 

1970- 71, 1971-72 
Admin: John Warner 
Facil :* Bill ie Vest 

ORANGE COUNTY COMMUNITY COLLEGE 
115 South Street 
Middletown, New York, 10940 
Tel. 914: 363-1121 

1971- 72 

Admin: Robert Greenman 
Facil: Russell King 
Herman Pollack 



PUTNAM-WESTCHESTER COUNTY BOCES 
845 Fox Meadow Road 
Yorktown Heights, New York, 10598 
Tel. 914: 245-2700 

1970- 71 

Admin: Walter Goodman 

Donald Bamford 
Facil : Pat Carlo 

Robert Wagner 

ROCKLAND COMMUNITY COLLEGE 
145 College Road 
Suffolk, New York, 01901 
Tel. 914: 356-4650 

1971- 72 

Admin: Seymour Eskow 
Facil: Henry Weinstock 

SUFFOLK III BOCES 

LA Wilson Technological Center 

Lindenhurst Annex 

301 Hoffman Avenue 

Lindenhurst, New York, 11757 

Tel. 516: 586-0100 

1970- 71. 1971-72 
Admin: John Grime 

John Burke 
Facil: Virgil Indelicate 
Janet Moses 

WESTCHESTER COMMUNITY COLLEGE 
75 Grasslands Avenue 
Valhalla, New York, 10595 
Tel. 914: 946-1616 

1971- 72 

Admin: Donald Mahoney 
Facil : Frank Crowley 

WILLIAMSVILLE CENTRAL SCHOOL 
DISTRICT 

Williamsville, New York,' 14221 

Tel. 716: 634-5300 

1970-71 

Admin: Ralph Mauro 
Facil : Ralph Mauro 
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PREFACE 



This Final Report is written in fulfillment of a requirement in the 
contract between the University of Massachusetts and the States of 
Massachusetts and New York by which The Evaluation Service Center for 
Occupational Education was established. The writing and organization 
of the Report have been undertaken in the spirit of providing complete 
information about ESCOE and its products in an accurate and easily 
readable form. 

Descriptive statements enclosed throughout this report are within a 
large block on the page. 



Amherst, Massachusetts 
June 1972 



I 



COPIES OF THIS FINAL REPORT ARE AVAILABLE 
UPON REQUEST TO: 



[in MASSACHUSETTS] 

Massachusetts Department of Education 
Director of Research Coordinating Unit 
Division of Occupational Education 
182 Tremont Street 
Boston, Massachusetts 02111 



[in NEW YORK] 

New York State Department of Education 
Director of Research Coordinating Unit 
Bureau of Occupational Education Research 
Albany, New York 12224 



Information contained within this Report and the 
other products of The Evaluation Service Center for 
Occupational Education (1970-72) are in the public 
domain. Their widespread dissemination and use is 
encouraged by those who have been involved in and 
were responsible for this project. 

-June 30, 1972- 
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PROJECT EVALUATIONS 



j An evaluation of the effects of training in writing behavioral objec- 

I tives was carried out by Kathryn A. Hecht in cooperation with ESCOE. i 

: The study is entitled "An Empirical Testing of Popular Expectations j 

' i 

I Held for Teachers Who Have Written and Used Behavioral Objectives" j 

i (School of Education, University of Massachusetts, 1972). The study \ 

! sought to empirically demonstrate popular beliefs concerning the | 
' benefits to teachers from the experience of writing ana using behavior- 

; dl objectives. 

i 

' ESCOE facilitators (identified by the pseudonym "Project Information" : 

' in the study) were cliosen as an objective-experienced sample and I 

j 

compared with a similar group of teachers fi'om Massachusetts without | 

[ such experience. The study will be available through Dis sertat ion j 

i I 

j Abstracts , University Microfilms, Ann Arbor, Michigan. i 

i i 

! I 

i ! 

j There is also an over-all evaluation of the ESCOE project in process 
at the University of Massachusetts which will be, available in the form 
of a doctoral dissertation from The Center for Occupational Education, 
School of Education by June 1973. 



I 
i 
( 
f 

! 
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INTRODUCTION 
BEHAVIORAL OBJECTIVES AND EVALUATION AT ESCOE 
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ESCOE PRODUCTS 
(as of June 30, 1972) 



RAW OBJECTIVES - 10,361 

SYNTHESIZED OBJECTIVES - 724 

CRITERION-REFERENCED TESTS - 4 

TRAINING PROGRAM - Content, Methods, Materials 

PROGRAMMED TEXT - A Self- Instruction Manual on Writing 

Behavioral Objectives 

CONFERENCES - Training and Planning, for Facilitators 
and Administrators, 8 

FIELD SERVICES & WORKSHOPS - On a Continuing Basis in 

the LEAs 

PUBLICATIONS - Instructional and Reporting Documents, 5 



BEHAVIORAL OBJECTIVES AND EVALUATION AT ESCOE 



A behavioral objective is a statement of precisely what a student 
will be able to do at the conclusion of a particular sequence of 
instruction, including the conditions under which he must perform the 
task and the criteria used to determine whether or not the perform- 
ance is mastered (learned). The following are examples of objectives, 
written by teachers and submitted to the ESCOE data bankJ 



I. " Given the values of all elements in a transformer 
coupled circuit, the student wi I I calculate a) the 
coefficient of coupling; b) the induced voltage in 
the secondary; c) the impedance reflected in both 
the primary and secondary at resonance, wifh 10% 
accuracy. "2 

2' " Given three negative stats and darkroom equipment, 
the student will produce one positive print having 
3 separate tones, in one hour."^ 



To make possible an orderly approach to the description of a course 
of study, a content analysis must be completed. A course must be 



^Underlined words in the objectives quoted here refer to the begin- 
ning of each of the three components of an objective: Conditions 
(Given), Performance ( the student will ). Extent (with, vn). 



^Category breakdown 
for this obj ective : 

Field of Study: Technical 

Major Group: Engineering Related 



Industry. 

Major Group: Commercial Art 
Occupations 



3Category breakdown for this 
objecti ye : ^ 

Field of Study; Trade & 



Technology 



Subgroup: Electrical Technology 
Block: Resonant Circuit Theory 
Unit: Coupled Circuits 



Subgroup: Commercial Art 
Block: Design 
Unit: Photography 
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BEHAVIORAL OBJECTIVES AND EVALUATION AT ESCOE (continued) 
i broken down into logical "blocks" of learning. These "blocks" must 1 

i 

I then be broken down into smaller pieces or "units," from which I 

; behaviorally stated objectives can be derived. For example: | 

' . j 

Course: Survey i ng I | 
Block: Transits j 
Units: I . Setting up the instrument i 

r 

2. Reading the horizontal circle j 

3. Reading stadia , 1 

I 

From these Units the actual teaching objectives are then constructed. | 

I The nunt>er of performance objectives derived from these Units will i 
j i 
I vary and could be one, two, three or twenty any number required ' 

j to fully describe that Unit. 

i 

The progression from the writing of individual objectives to the ! 

1 r 

process of evaluation is a complex one and involves several stages. 

I 

! After it was decided that the use of behavioral objectives would | 

i 1 

serve as the basis for the statewide evaluation system, the first | 

step was to generate the objectives. Only when a large number of | 

i 

objectives had been written could the creation of evaluation pro- ; 

i 

cedures actually take place. Consequently, much of the ESCOE effort i 

i 

was directed towards constructing a system of instructional and | 

i 

support services within which individual teachers in schools offering j 

I 

occupational education would write objectives for ESCOE. Thirty-six j 
secondary schools and community colleges in Mass-^ohusetts and New i 

J ^ _ j 
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BEHAVIORAL OBJECTIVES AND EVALUATION AT ESCOE (continued) 

' York have participated in the project, 1970-72. Approximately 

eighty facilitators in these LEAs have directed over 1000 teachers in 
the writing of objectives. 

t 

! 

Over tin thousand separate objectives in thirty-eight different Sub- j 
I groups have been written by these teachers during the two years of 

the project- Naturally there was considerable near-duplication, as 
I well as considerable diversity, in objectives written for the same 

i 

tasks by different instructors in different LEAs, A large number of 
procedures can be involved in the task of "reading stadia'' for 
instance, even though this task is essentially the same everywhere 
and the number of alternative procedures is finite. 

i 

Thus ESCOE found itself faced with the necessity of consolidating, 
for many tasks, a number of similar objectives into one objective | 
that would incorporate within its basic demands the various methods 
and conditions which individual instructors had submitted as repre- 
sentative of their own students and teaching- These final objectives | 
are called "synthesized objectives" — SYNOBs and the alternative \ 
procedures within them are referred to as "form changes." ; 

i 

The process of synthesis also became necessary in order to take 
steps towards the ultimate goal of instituting a statewide evaluation: 
system- The SYNOB was designed to provide both a means by which ] 
schools could select objectives on which their students would be ' 

-5- 



BEHAVIORAL OBJECTIVES AND EVALUATION AT ESCOE (continued) 

tested, and also by which these same schools and teachers could 
follow their own preferences in the processes of instruction, 

i The next step towards the creation of an evaluation procedure was i 
the construction of tests from the collection of synthesized objec- 
tives. These tests reflect both the common objectives across 
schools and the individual variations within schools that are reflec- 
ted in the form changes. A test in a year's course in Cabinetmaking 
and Millwork, for instance, is so constructed that a teacher will 
select from a range of test items in that occupational area, only 
those items that were actually taught as objectives in his class 
that year. A student in any class is tested only on objectives that 
he actually performed. Ordinarily there will be a sufficient number 
of comiTion objectives among different LEAs that test results will be 
comparable among them. 

The results of the tests in their totality and through item 
analysis will make possible the evaluation of the effectiveness 
of methods and goals of teaching in a classroom, in a school, in the 
local and statewide system of schools. Evaluation will, in effect, 
measure learning. If this process of evaluation is continual, as it 
is intended to be, intelligent and directed changes can be brought 
about id the educational systems. These changes have as their pur- 
pose the improvement of education for the sake of the student. 



OCCUPATIONAL AREAS WITH OBJECTIVES IN DATA BANK 



USOE OCCUPATIONAL 
CODE PROGRAM 

010000 Agriculture 

030000 Business 

040000 Distributive Educ. 

050000 English Language Arts 

070100 Dental 

070200 Medical Laboratory 
Technology 

070300 Nursing 

080300 Physical Education 

090000 Home Economics 

110000 Mathematics (Calculus, 
General Math, Geometry) 

130000 Natural Sciences 

140100 Accounting & Computing 

140200 Business Data Processing 

140300 Filing, Office Machines, 
General Office Clerical 
Occupations 

140600 Personnel. Training, h 
Related Occupations 

140900 Typing & Related 
Occupations 

150800 History 

160104 Automotive Technology 

160106 Civil Technology 

160107 Electrical Technology 

160108 Electronics Technology 
160113 Mechanical Technology 
150114 Metal urgical Technology 
160117 Scientific Data Processing 

O -7- 
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USOE 


OCCUPATIONAL 


CODE 


PROGRAM 


170100 


Air Conditioning-Cooling 


170102 


Air Conditioning- 




Heating 


170201 


Electrical Appliance 
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Fender 


170302 


Automotive Mechanics 


173100 


Small Engine Repair 


170700 


Commercial Art Occupa- 




tions 


171004 


Masonry 


171005 


Painting & Decorating 


171007 


Plumbing & Pipefitting 


171010 


Roofing 


171300 


Drafting Occupations 


171400 


Electrical Occupations 


171500 


Electronics Occupations 


171900 


Graphic Arts Occupations 


172302 


Machine Shop 


172305 


Sheet Metal 


172306 


Welding & Cutting 


172602 


Cosmetology 


172901 


Baker 


172902 


Chef/Cook 


172903 


Meat Cutter 


173300 


Textile Production & 




Fabrication 


173500 


Upholsteri ng 


173601 


Cabinetmaking & Millwork 


171001 


Carpentry 



KEY TERMS 
USED IN THIS REPORT 



ESCOE 

Evaluation Service Center 
for Occupational Education 

BEHAVIORAL OBJECTIVE 

Statement describing, in observable 
terms, what a student will be able 
to do after completing a prescribed 
unit of instruction 

RAWOB 

Raw objective; a behavioral objective 
before synthesis 

SYNOB 

Synthesized objective; raw objectives 
that are similar in performance combined 
into one statement 

LEA 

Local Educational Agency, a school or 
group of schools operating under one 
administrative body. 
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THE THREE COMPONENTS OF A BEHAVIORAL OBJECTIVE 
DEFINITIONS AND EXAMPLES 



CONDITIONS 



The exact circumstances, e,g. materials 
and procedures (or restrictions on them), 
und^er which an objective is performed 

Given the values of all elements 
in a transformer coupled cfrcuit, 



PERFORMANCE 



The exact observable behavior a student 
who has mastered the requirements of the 
objective will manifest. 

the student will calculate a) 
the coefficient of coupling; 
b) the induced voltage in the 
secondary ; c) the impedance 
reflected in both the primary 
and secondary at resonance, 



EXTENT 



The criteria--quality , tolerances, accuracy, 
percentage of correct answers, etc. --used to 
measure the performance. 

/ 

with 70^ accuracy. 



CThis objective belongs to the 
subgroup Electrical Technology.] 



CATEGORY BREAKDOWN: DEFINITIONS AND EXAMPLES 



Every objective written for ESCOE has bec^n classified according 
to this category breakdown for occupational programs. Objectives 
are written within a "Unit." 



FIELD OF STUDY 



Broadest category of occupational 
classification; e.g.. Trade & Industry, 
Health Occupations^ etc. 

Techn ical 



MAJOR GROUP 



SUBGROUP 



BLOCK 



MAJOR TOPIC 



UNIT 



Classification of occupational programs 
within a Field of Study; e.g., within 
Health Occupations: Dental Services, 
Medical Services, Nursing, etc. 



Engineering Technology 



Occupational prograins within a Major 
Group. 

Civil Technoiogy 



Largest instructional segment within 
a Subgroup. 

Elementary Surveying 

(Post-secondary only) 

A division within a Block, created for 
the particular characteristics of post- 
secondary programs. 

Tap i ng 



Instructional segment within a Block or 
Major Topic. 

Correci ion for \<)n<i 
or short tape 



The number of objectives written to completely describe 
a Unit may vary, depending on the tasks, skills, or operations 
necessary for mastery of the Unit. The performance of every 
task within the Unit must be stated in the objectives so 
that no task is untapped. 



II 

ESCOE: HISTORY AND OVERVIEW 
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OPERATIONAL GOALS OF ESCOE 



TO DEVELOP A BANK OF BEHAVIORALLY-STATED 
OBJECTIVES IN SELECTED AREAS OF STUDY 

TO TRAIN FACILITATORS AND TEACHERS 
IN PARTICIPATING LEAs TO DESCRIBE 
SELECTED AREAS OF THE CURRICULUM BY 
BEHAVIORAL OBJECTIVES 

including training in 

the rationale, uses, and implementation 
of objectives. . . 

writing... 

classifying. . . 

editing. . .and 

synthesis... 

of BEHAVIORAL OBJECTIVES 



TO DEVELOP A BANK OF TEST ITEMS AND 
PROCEDURES DIRECTLY RELATED TO OBJECTIVES 
IN THESE SELECTED AREAS OF STUDY 

TO CREATE A COMPUTER INPUT AND 
FEEDBACK (RETRIEVAL) SYSTEM 

TO DEVELOP A MODEL FOR SYNTHESIZING 
OBJECTIVES 

TO CREATE A FEEDBACK LOOP BETWEEN 
THE LEAs AND ESCOE 



ESCOE Gi'-i delines and Approach 



ESCOE GUIDELINES 



ESCOE is a component of a statewide systems approach to meeting 
program evaluation needs in each state. The ESCOE produce pro- 
vides a data base for decision-making in education. 



ESCOE is an information feedback system for curriculum design and 
modification, which helps to diagnose strengths and weaknesses 
of existing programs, and to evaluate content for relevance. 



Program objectives are determined by LEAs, not prescribed by 
central authority. Program evaluation is responsive to differences 
within and between local instructional programs. Diversification 
viewed as the most promising route to program improvement. 



ESCOE APPROACH 



ESCOE is designed to act as a neutral agent to feed back program* 
evaluation information on a continuing basis to managers of 
occupational education on all levels within participating states. 



It also provides an alternative to standardized testing by 
measuring program achievement in terms of locally stated objectives. 



ESCOE utilizes the instructional and technical capabilities of 
experienced school personnel in operating the system. 



The ESCOE product supplies data for increased accountability focused 
on student achievement at state and local levels, but not at the 
cost of local autononiy or sound educational practice. 



ERLC 
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SERVICE COMMITMENT WITH LOCAL EDUCATIONAL AGENCIES 



Staff at The Evaluation Service Center for Occupational Education 
felt it important that the responsibilities and goals of both ESCOE 
and the LEAs be clearly stated and recognized by all involved with 
the project. This "Statement of Intent," as it was originally 
called, was written and disseminated in September of 1971. 

ESCOE' S COMMITMENT 

The Evaluation Service Center recognizes the necessity for improved 
and expanded services available to the LEAs. ESCOE's success depends 
upon the quantity and quality of the services it supplies. It is 
crucial that the ESCOE services are broadly based and readily avail- 
able. The list of services that follows is subject to modification 
according to the needs of participating LEAs. 

1. Conduct orientation conference for administrators of prospective 
LEAs and of state departments of education. Administrators of parti- i 
cipating LEAs and state departments will be systematically Informed j 
of the progress of the project. | 

I 

1 2. Conduct training conferences for facilitators in order to present \ 
I the materials and expertise needed to develop ESCOE's system in the ; 
I LEAs. Conduct other conferences as needed for dissemination and test i 
i administration. The' expenses for such conferences, except for travel J 
I will be borne by ESCOE. | 

i 

j 

I 3. Conduct initial teacher-training conferences in LEAs and follow- 
i up Workshops as necessary. 



SERVICE COMMITMENT (continued) 



4. Maintain frequent visits to LEAs by ESCOE field staff for the 
purpose of assisting teachers and facilitators in writing and 
selecting objectives and in completing the reporting forms- 

5. Provide all forms and training materials necessary for full 
participation. 

6. Maintain readily available communication (mail and telephone) 
between LEAs and ESCOE. 

7. Maintain an editorial process to receive and feed back comments 
on reporting forms and other activities of LEAs. 

8. Deliver computer printouts from data bank as requested by LEAs. 

9. Develop and administer performance tests, 

10. Feed back criterion-referenced test data as requested by LEAs. 

11. Facilitators and teachers will not^ be paid by ESCOE for the 
services they provide within their LEAs. Occasionally ESCOE will 
utilize staff from participating LEAs for such tasks as synthesizing 
objectives and development of criterion test materials. In these 
cases, ESCOE will compensate the teacher directly from ESCOE consul- 
tant funds. 

j LEA^S COMMITMENT 

I From the experience gained during the first year of operation, it is 
clear that modification of the process for developing the ESCOE 
system in the LEAs is necessary. The conditions described here are 
derived from observations by the ESCOE field experiences and from 



SERVICE COMMITMENT (continued) 
statements submitted by LEA facilitators. 



1. The primary condition for participation should be that the LEAs 
believe in the philosophy of the ESCOE project as stated in the 
Plannin g Document , namely, "that American society and youth are 
best served if program objectives for occupational education are 
selected by LEAs and not centrally prescribed by state departments." 
A belief in this philosophy and the realization that participation 
in the ESCOE project will involve, the LEA instructional staff in 
describing their programs in clear, specific, measurable statements 
should convince the LEA administration that the fruits of the ESCOE 
effort will be worth the labor. 

2. Each LEA must assign a minimum of two "facilitators" to act as 
coordinators within the LEA. The facilitators must attend all 
ESCOE training and dissemination conferences. They will learn the 
philosophy, operation, benefits, application, outcomes, etc. of the | 
ESCOE system as well as how to write behavioral objectives. Their I 
function will be to instruct and coordinate the writing and selection! 
of behavioral objectives in their LEA, and also to coordinate the ! 
test administration along with the ESCOE staff. The number of j 

i facilitators above two should be contingent upon the structure and j 
size of the LEA. If a BOCES LEA has three separate schools, then | 
probably one facilitator in each school would work out well, whereas ! 

I two facilitators in a regional or local vocational school might be ! 

I best. Community colleges should have at least one facilitator at i 
j 1 
j each campus if the buildings are at different locations. i 

i i 
i 3. Experience has shown us that in some situations administrators 

i do not have the time for direct involvement with teachers in this 

i 

, task. Thus, we are recommending that the ideal team for facilitation 
\ of well -written behavioral objectives in the LEA be one teacher and 

; one coordinator/supervisor/administrator or a combination thereof. I 

! 1 



SERVICE COMMITMENT (continued) 



However, it is imperative that the facilitator be a patient and 
empathetic person who is willing to devote much time and effort to 
the assignment. 

4. Released time for the facilitators and teachers has proved to be 
the most effective method for full participation. The time- 
availability factor varies greatly across schools, so no one formula 
could apply in all situations. A lighter teaching load throughout 
the year would seem appropriate for facilitators. The equivalent of 
one period per day should be adequate for this chore. Participating 
teachers must attend a minimum of four after-school meetings for 
training by facilitators and ESCOE staff. In addition, approximately 
five departmental -teacher meetings might be necessary to write and/or 
select objectives. Released time for teachers, if necessary, cou^d 
be accomplished by the use of substitute teachers, 

5. By either writing or selecting objectives, or both, each LEA 
must completely describe the subgroups taught in that LEA, This 
means that LEA instructors in these subjects will submit or select 
all the behavioral objectives in their instructional program, which 
means all grade levels and all Blocks and Units therein. All sub- 
jects in an occupational curriculum are welcome. Concentration 
(synthesis and test development) is limited by economic factors with- 
in the project budget; however, it ■>$ inherent in the ESCOE philos- 
ophy to have all occupational education program objectives developed 
to the highest degree possible and In as many dimensions as might be 
useful . 
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6. Objectives and other data must be submitted on the proper 
reporting form (original copy) In easily readable print, preferably 
typewritten. If typewritten objectives are not possible, then they 
are acceptable only in clear block-print. To emphasize the import- 
ance of legibility, It must be rem embered that these objectives are 
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SERVICE. COMMITMENT (continued) 

read by card-punch operators who usually are not the least bit 
familiar with the descriptive terminology before them. 

7. Administrators must support facilitators in providing adequate 
meeting time, space, supplies, and ancillary services to facilitate 
well-written, completely representative objectives. Scheduling 
meetings with teachers and making clerical staff available are 
crucial adjuncts to the function of the facilitator. 

8. Facilitators will assist teachers in submitting well-prepared 
reporting forms. This means reading and editing teachers' objectives 
and feeding them back to the teachers if problems exist. An 
objective should not be submitted to ESCOE except in acceptable form. 



September 1971 



ESCOE: INCEPTION OF THE PROJECT 



ESCOE is the fruition of more than two years of planning and 
pilot efforts. The impetus, as well as the funding source, 
for the development of an information feedback system for voca- 
tional education (as it was originally called) came from federal 
legislative action calling for program evaluation. This mandate 
has been applied to almost all types of sponsored education, and 
was specifically formulated for occupational education in 1968, 
under the amendments to the Vocational Education Act of 1963.* 



The Massachusetts Occupational Education Research Coordinating Unit 
chose to develop a systems approach to evaluation, to be based 
on program assessment, using locally derived behavioral objectives. 
It proposed a system that is based on current methodological 
principles but which has not been put to the test of practical 
usage in any area of education. The establishment of ESCOE was 
designed to test the feasibility of operationalizing a statewide 
systems approach to evaluation. The inclusion of New York 
as the second participating state in ESCOE increases the general - 
izability of the system by widening the vc^-^iety of programs the 
system will accommodate. The Evaluation Service Center is a 
prototype designed for expansion within member states and to other 
states . 



^Public Law 90-576 5 Amendments to the Vocational 
Education Act of 1963, states that the State Advisory 
Committee shall ''evaluate vocational education programs, 
services, and activities assisted under this title and 
publish and distribute the results thereof: and prepare 
and submit an annual evaluation report. . .which (1) 
evaluates the effectiveness of vocational education 
programs, services, and activities carried out in the 
year under review..,.'* 



A CHRONOLOGICAL OVERVIEW OF ESCOE^S ACTIVITIES 
SCHOOL YEAR I, 1970-71 

In October 1970 the contract was signed by the states of Massachusetts 
' and New York and the University of Massachusetts at Amherst estab- 
; lishing The Evaluation Service Center for Occupational Education until! 
I June 30, 1972. Ideas for such an agency had been discussed for two \ 
; years, and already in September of 1970 administrators and facilita- i 
tors from ten schools In each state had met with the Research Coord- 
inating Unit Directors from the two states and with the Director of 
the Center for Occupational Education at the University to make 
specific plans for the project and a commitment to it. 

In November 1970 the first facilitators' conference under the 
auspices of ESCOE itself was held in Amherst for the purpose of 
training the facilitators in the skills of writing behavioral 
objectives and in the procedures for transmitting those objectives 
to the ESCOE office. At this conference LEA personnel, ESCOE staff, 
and state department representatives established many of the opera- 
ting procedures. Working Paper No. 1 (December 1970) followed up 
this conference with a detailed explanation of the classification and 
coding of objectives. In January 1971 a second conference was held 
to further develop the facilitators' skills in dealing with objectives 
and to make final plans for the coming year. 

This month, January 1971, also marked the situating of The Evaluation 
Serv ic e Center in offices at 85 North Whitney Street in Amherst^ Ma, 
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CHRONOLOGICAL OVERVIEW (continued) 

I Facilitator.; had been trained in writing objectives at the first ' I 
i j 
I conference in Novertber and at the January conference; the first order ' 

i ; 

1 of business was to continue the training and advising of facilitators \ 

t 

i in their own LEAs and to support teachers who were writing objectives.; 

i One staff member traveled to the LEAs for this purpose two to three ' 
days a week during February, March, and April. At The Evaluation | 
Service Center the system for reception and processing of objectives j 
was further evolved and a computer system into which all objectives 
were to be stored was developed. Refinement of both the coding end 
computer system has been continual for the duration of the project. 

With the increasing number of objectives in many different Subgroups 
it was decided that four Subgroups -- Machine Shop, Industrial 
Electronics, Millwork & Cabinetmaking , and Automotive Mechanics -- 
chosen because of the large student enrollment in each area would be 
used as a prototype for synthesis and test development,. Input of 
objectives in all Subgroups was encouraged, however, and synthesis 
in twenty- two more Subgroups took place during the Spring of 1972. 
In March 1971 Technical Report No. 1 was published, superseding 
Working Paper No. 1 and elaborating reporting and classification 
procedures. 

When concentrated attention was given to objectives in the four 
Subgroups to which ESCOE would devote its first-year efforts in 
synthesis, it became obvious that a further refinement of the course 
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CHRONOLOGICAL OVERVIEW (continued) 

classification schemes was necessary. For placing a particular Sub- 
group within a larger instructional phase, codes from the United 
States Office of Education had been used and found satisfactory. 
However, a breakdown of instructional phases within a Subgroup did 
not exist in the USOE codes, and instructors in the LEAs were 
classifying the objectives they wrote by schenies too dissimilar 
to be useful in dealing with large numbers of objectives. According- 
ly, in May of 1971 two instructors in each of the first four 
Subgroup areas met with the ESCOE staff to devise, from their own 
knowledge of the fields and from suggested breakdowns sent in by 
the LEAs, a standardized Block and Unit breakdown. Writers of 
objectives were encouraged to use these breakdowns but were also 
advised to suggest modifications when those already in existence were 

not suitable for their needs. Since that time. Blocks and Units for e 

' I 

total of thirty-eight Subgroup areas have been devised and 
disseminated. 

i 
I 

Also in May 1971 a Spring Dissemination Conference at Cape Cod was j 
held to review the year's activities, to plan for the following ! 
school yfiar, and to give facilitators computer printouts of the raw \ 
objectives written by their LEAs. 

The next major phase of the ESCOE effort was the synthesis of objec- ! 
tives in the first four sel ected Subgroups- In July the ESCOE staff 
met with Dr. Jimmie Fortune who introduced them to the process of j 
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CHRONOLOGICAL OVERVIEW (continued) 

synthesis,- as developed by David Berliner, and then, briefly, with 
Dr. Berliner. Two facilitators from each of the four Subgroup areas 
were tapped to learn synthesis from the ESCOE staff and then to 
actually synthesize objectives in the four areas. These synthesized 
objectives were entered into the data bank in October and printouts 
were given to facilitators at the conferences in October and 
November , 1971 . 

The Expanded Planning Document for School Year 1971-72 was submitted | 
by ESCOE to the Associate and Assistant Commissioners for Occupation- 
al Education in Massachusetts and New York, respectively. This 
document detailed plans for budget, staff, and scheduling for the 
fiscal year 1972. A successful effort was made to include a number 

i 

of post-secondary schools in the project for its second year» | 

) . I 

SCHOOL YEAR II I 

I 

During the second school year of the ESCOE project, there were eleven 
participating LEAs, secondary and pert-secondary, in Massachusetts 
and fifteen in New York» In September, two staff members were 
assigned primarily to field services for the LEAs, spending approxi- 
mately three days a week at the LEAs and two in the ESCOE office in 
Amherst. The Administrators' Conference was held in this month, 
emphasizing planning and commitment for the school year. 

i 

The generation of synthesized objectives, now an important aspect of 
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CHRONOLOGICAL OVERVIEW (continued) 

ESCOE's efforts, required a complex system of feedback between ESCOE 
and the LEAs in order that the development of the tests might accu- 
rately reflect course content. The Instruction Manual: Synthesized 
Objective Package , published in October of 1971, was designed to make 
this process as efficient and uncomplicated as possible. In addition, 
A Behavioral Objective Training Package: For Facilitators , incorpora- 
ting techniques of training developed by ESCOE, and reporting proce- 
dures, was designed particularly for the new facilitators who would 
be joijring the project in the Fall of 1971. With minor revisions and 
changes, this publication was used as the primary instrument for 
training teachers in the LEAs to write objectives. 

The Training Package was used at the First Year Facilitators' 
Conference in November and by both first- and second-year facilitators 
as a resource for training teachers in their LEAs. Second-year 
facilitators' attended a conference in October and received intensive 
training in principles and methods of synthesis. Transfer of the data 
bank to Massachusetts State Department of Education facilities took 
place during November. On February 23 and 24, 1972, mid-winter 
vacation for most LEAs, a major Synthesizers' Workshop was held to 
train facilitators and teachers in synthesis. From this time until 
the end of June synthesis was carried out in the individual LEAs by 
those who had volunteered to carry out this work. Two ESCOE staff 
members spent raost of their time in the LEAs, giving advisory and 
refresher workshops in the writing and synthesis of objectives. 
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CHRONOLOGICAL OVERVIEW (continued) 



Field testing of machine shop tests was begun in March, as was the 
development of the Blocks and Units necessary for synthesis. A copy 
of the punched computer cards was ty "^nsferred to the computer at 
Hudson Valley Community College during April, Plans for the writing 
of a programmed text in writing behavioral objectives were finalized 
during this month; the text was completed in June. 

The Cabinetmaking & Millwork^ test underwent field testing in May. 
Also in May, the Spring Dissemination Conference was held for all 
facilitators where the primary emphasis was on the future possibili- 
ties for the work ESCOE had begun. Two filmstrips> including audio 
tapes, were being developed for ESCOE and are expected to be 
finished soon. These filmstrips, about twelve minutes each, are 
introductory in nature, explaining the need for evaluation and goal 
claHfication in education and the basic strategies of curriculum 
analysis using behavioral objectives. Three ESCOE staff members 
conducted a three-day workshop as part of a longer workshop 1n 
systems analysis for teachers at one of ESCOE's participating 
community colleges. The field staff also concentrated heavily on 
finalizing synthesis activities in the LEAs. The synthesized 
objectives will be entered into the data banks of each State 
Department of Education, 



1 



In the section of this Report on testing, Cabinetmaking and Millwork ^ 
IS referred tc as "Woodworking." i 
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CHRONOLOGICA:'. OVERVIEW: THE FUTURE 

r 

1 Activities very similar to those of ESCOE, 1970-72, will be carried 
j out at The Eva'^uation Service Center for Occupational Education to 

j i 
: be based at Hudson Valley Community College in Troy, New York, from i 

' i 

I July 1 , 1972, to June 30, 1973, A staff member from the present 

■ i 

ESCOE will direct the new ESCOE full time and will have the assistanc^ 

! I 

; of one half-time profes:ii onal staff person. The emphasis of this ! 
I i 
I project will be on the implef7)entation of behavioral objectives, in I 

! instruction and in testing, and on curriculum development. At the j 

I start of the project twenty-five teachers in three secondary and 

j 

1 six post-secondary LEAs are committed to the project. \ 

i 
I 
} 

In Massachusetts developnrent work on the statewide eva'^uation system, 
of which ESCOE was Phase I, will continue at the Management cjnd 
Information System Project (MISOE). Further information may be 
obtained from the Division of Occupational Education, Massachusetts 
Departrent of Education, Winchester, Massachusetts 01890. 
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THE ESCOE TRAINING PROCESS 
INTRODUCTORY VIEW: OBJECTIVES, PROCESS, PRODUCT 

OBJECTIVES 

Train facilitators 
Train teachers 

Feedback of objectives to ESCOE 
PROCESS 

Conferences (for facilitators, in central locations) 

On-site Introduction tu ESCOE & Systems Approach (one hour in 
the LEAs) 

Workshops in LEAs on writing behavioral objectives, synthesis, 
and procedures for submitting objectives 

Objective Writing: by faculty in LEAs for their own courses 

Raw Objective Dissemination: Printouts to LEAs on request 

Training for and Development of Blocks and Units: by selected 
and volunteer subject area specialists (teachers, department 
heads, etc J 

Synthesis Workshop: two subject area specialists for each Subgroup 
to be synthesized 

Writing of SYNOBS: by those who participated in the Synthesis Workshop 
Dissemination of SYNOBS: to all LEAs 

PRODUCT 

80 facilitators trained 

1000-1500 teachers trained 

10,361 RAWOBs in 38 Subgroups produced 

734+ SYNOBS in 26 Subgroups produced 
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THE TRAINING PROCESS 



A major thrust of ESCOE's efforts, and, in fact, the one upon which 

all ESCOE's activities have depended, has been that directed towards 

training facilitators in the LEAs in the writing and editing of 

behaviorally-stated objectives. Facilitators, once py^acticed and 

knowledgable, then trained teachers in their LEAs to write and edit 

objectives for their own courses. In this way, eighty facilitators 

[ and between 1000 and 1500 teachers were trained in the ESCOE theory 
i 

\ and technique of writing objectives. In addition, approximately 
j 

I thirty-five facilitators during the two years of the project were 

S 

J trained in and carried out synthesis of raw objectives. 

I Facilitators were trained primarily at conferences held for that 
I purpose once a year and subsequently at refresher workshops in the 
LEAs. Field services to the LEAs by ESCOE's two Coordinators for 
Secondary and for Post-Secondary schools have been a most important 
means by which both training and mutual feedback between ESCOE and 
the LEAs has been accomplished. Such services included introductory 
workshops on ESCOE and behavioral objectives, longer workshop 
sessions with practice in goal analysis and the writing of objectives, 

j delivery of supplies and training materials to the facilitators, and 

i 

assistance to the facilitators in editing objectives and in conduct- ; 
ing workshops in their own LEAs for involved faculty. I 

I i 

The two field staff persons held at leart tv^/G workshops a week in j 
the various LEAs. and often more. These works ho ps were held at the ! 
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THE TRAINING PROCESS (continued) 



j request of the LEA. The length of the sessions varied from two hours 

i 

I one day after school to a full two days, depending on the time made 

j available by the LEA. Larger 'Vegional" workshops were held approxi- 

i 

j mately once a month. When synthesizing was begun, workshops on 
synthesis were held also. 

Because the success of the ESCOE training process was so vital to the 
accomplishment of all of ESCOE's goals, much effort went into the 
development of training materials and approaches. The Behavioral 
Objectives Training Package , intended as a complete manual for all 
ESCOE facilitators, and later, for the use of teachers in the 
LEAs, was one such result. Some of the exercises from the Training 
Package are reproduced in the following pages. Staff m^;,ibers also 
developed three series of overhead transparencies with accompanying 
narration for use in their workshops. These series covered the '■ 
topics of "An Introduction to ESCOE and Behavioral Objectives," "An 
Overview of Systems, Goal Analysis, and Objectives," and "Synthesis." ; 
Selected materials from these transparencies are also reproduced in 
this section as samples of the training content. 

Filmstrips from The Instructional Objectives Exchange in California 
were bought and used, and another filmstrip on objectives 1s in process 
for ESCOE by a student at the University of Massachusetts. A program- 
med text on writing objectives was written and was printed in June | 
1972. Copies of this text will be available as a separate appendix. j 
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THE TRAINING PROCESS (continued) 

Training, in whatever mode--workshops , publ ications\ visual presenta- 
tions--emphasized the background and philosophy of ESCOE--its purpose 
as a center, its reason for being, its relationship to the LEAs--as 
well as overviews of a systems approach to education, including 
thorough instruction (generation, rationale, uses, formats) in writing 
behavioral objectives, and thorough introductions to synthesis and 
to testing and evaluation. 

; The ESCOE staff came to see behavioral objectives as components of a 

i total systems approach to education; this approach, as well as the 

; writing of objectives, received emphasis in the training protocols, 

i In fact, two courses, given through the University of Massachusetts 

I Division of Continuing Education, were developed for those facilita-: 

i 

■ tors who wished to receive credit for their work as ESCOE facilita- 

: tors and to further explore the possibilities of such involvement. 

j 

i These courses emphasized the integration of objectives writing with 

I 

I the systems approach to education and are described in this section 

I 

j of the Report. 

i 

j 

In their work with facilitators and teachers the ESCOE staff adhered 
I to the belief that each LEA existed with its own unique autonomy and 
i significance. Facilitators and teachers committed to the ESCOE pro- 
; ject did not simply produce objectives for ESCOE but were offered an 

intense exposure to the possibilities in curriculum development that 

may be gained through systems analysis and the imaginative use of 

behavioral objectives. 

-31- 



TRAINING OF FACILITATORS AND TEACHERS 
AN OVERVIEW IN OUTLINE FORM 



PURPOSES OF THE TRAINING 



Facilitators 



Teachers 



To prepare facilitators to: 



To prepare teachers to fully 
describe their courses in 
terms of behavioral objectives 
in a format and of a quality 
acceptable to ESCOE. 



1. teach faculty in their LEAs 



to write and edit objectives 
2. organize and coordinate the 



process of preparing objectives 
in each LEA and to support the 
efforts of those writing 
them by informing administra- 
tors and teachers of the uses 
and advantages of objectives 
and of the LEA*s association 
with ESCOE 

3. edit objectives before sending 
them to ESCOE. 



Lectures and panel presentations by ESCOE staff & guest speakers 
Filmstrips 

Overhead transparencies series, with narration 

Small group exercises in writing, editing, and synthesizing 

objectives, using the Behavioral Objectives Training Package 
Group discussions among participants and ESCOE staff members 



GENERAL CONTENT COVERED DURING TRAINING PROCESS 

Systems Approach to Education; Goal Analysis and Task Analysis 

ESCOE: philosophy, inception, purposes, operations 

Behavioral Objectives: definition, rationale and uses, how to write 

Synthesized Objectives: as for behavioral objectives 

Testing and Evaluation: relation to objectives, methods, purposes 



TRAINING STRATEGIES 



Facil itators 



Teachers 



Accomplished primarily at the all- 
facilitator conferences held periodi- 
cally at a central location. 



Accomplished primarily at work- 
shops in individual LEAs, given 
by facilitators and/or ESCOE staff. 
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TRAINING OF FACILITATORS AND TEACHERS (continued) 



OBJECTIVES OF TRAINING 



Separate objectives were established for each workshop and conference. 
The following list was used at a curriculum development workshop and 
may be considered typical of more recent training sessions. 



After completion of the workshop each active participant 
will be able to do the following: 



Given a diagram of an educational system, describe the 
function of each component. 

Given a goal and using the strategy of operational izing 
fuzzy concepts, derive performance statements describing 
this goal. 

Given a job, analyze and identify the necessary tasks 
required to successfully complete them. 

State the three components of a behavioral objective, and 
describe the function of each component. 

Recognize a behavioral objective by identifying its three 
component parts. 

Given a performance statement, construct a behavioral 
(measurable) objective that contains all of the necessary 
components. 
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GUIDE TO REMAINDER OF PART III: THE TRAINING PROCESS 



The remainder of this section consists of extracts from several 

of the modes of presentation used by ESCOE in its training programs. 

They appear in the following order: 

"INTRODUCTION TO ESCOE AND BEHAVIORAL OBJECTIVES" - Narration for 
Overhead Transparencies 

THE SYSTEMS APPROACH TO EDUCATION - List of Components 

"HOW BEHAVIORAL OBJECTIVES ARE EVOLVED BY CURRICULUM DEVELOPERS 
OR TEACHERS" - Extract from an Introductory Presentation 
Shown on Overhead Transparencies 

"OBJECTIVES: DEFINITIONS AND EXERCISES" - Exercises from the 
Behavioral Ob j ectives Training Package 

"TYPICAL AGENDAS FOR TWO TYPES OF ESCOE WORKSHOPS" 

ESCOE CONFERENCES r Overview and Agendas 

"COURSE CREDIT FOR ESCOE FACILITATORS" - Course Descriptions 
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"INTRCDliCTION TO IZlZ'- AhD BEHAVIORAL OBJrCTIVES" 
Narration for Ov'^'^;^t^^^J I v-jfiSQarenci es 



This narration ( or sonietinies an c-wp !ana::ion of what is on a slide) is one 
example of the training protocol, i iiusrrating the type of information 
presented in an introductory session, its order of presentation, etc. 



Slide / Narration or Explanacior of Slide 

# _ _ _ 



1. ESCOE - Acronym for Sl ide 2 

2. Evaluation Service Center for Occupational Education - What is it? 

3. A Catalyst - While nothing reel ly new is involved in this portion 
of ESCOE's approach, it i3 r^ossible that it could be the vehicle 
that could bring together nicke operational the many facets of 
educational systems — objectives, teaching methods, teaching 
materials, media and evaluation techniques. 

4. A Training Center - ESCOE, with a strong emphasis on service, is 
geared to provide a great 6f^b'l of training for the faculties of LEAs* 

5. A Data Collection Agency - Ail materials received by ESCOE will be 
coded, classified and entered in a computer data bank for retrieval 
under many different needs. 

6. A Designer of Evaluation Instruaients - After sufficient data have 
been collected in a given area of study, ESCOE, with the help of 
LEAs, will design and construct evaluation instruments. 

7. A Unifying Agent - A great need exists for unification of the many 
levels of education. A center such as ESCOE that could ultimately 
serve all these levels may possibly be the agent that could bring 
this about. 

8. Its Inception - ESCOE was brought into being because of a law. 

9. The Law ~ stated on Slide 10. 

10. Statement of Law - This law generated much thought as to how 
evaluation could be accompl isheo. Standardized testing was 
quite naturally one answer but was readily rejected. 

11. A Need - This rejection of standardized testing brought about the 
need for an alternate .-nethod of evaluation. Thus the concept of 
ESCOE was born. The question was asked, "Why not let each LEA 
specify its own objectives and be tested on these in lieu of a 
standardized curriculum? 

12. Its Philosophy explained in Slide 13 

13. That it is much better for an LEA to specify its own objectives 
and then be tested upon these. 

14. ESCOE's Role and Responsibility is spelled out in Slide 15. 

15. ESCOE, with the help of the LEAs, will endeavor to fulfill its 
responsibility through a four-pronged attack. To operational Ize 
the entire concept all four facets will be active concurrently 



NARRATION FOR OVERHEAD TRANSPARENCIES (continued) 



16. Facet I - Develop a Bank of Behavioral Objectives 

The LEAs will appoint facilitators from their faculties to 
act as liaison with ESCOE. These facilitators will be trained 
by ESCOE to function as resource personnel and to assist the 
faculty in writing behavioral objectives. 

17. Facet II - Create a Feedback Loop 

LEAs will feed to ESCOE objectives, methods, lists of teaching 
materials, consultants, while ESCOE will feed to LEAs training 
personnel, training materials, testing instruments, etc. 

18. Facet III - Develop Test Instruments 

With the help of consultants from LEAs and testing specialists, 
ESCOE will develop testing instruments for usv^ in the LEAs. 

19. Facet IV - Create an Ongoing Service Center 

ESCOE will continue to assist LEAs through supplying training 
personnel, training materials, printouts of objectives, lists 
of teaching methods and materials, etc. 

20. Why Behavioral or Performance Objectives? is dramatically 
answered by Slide 21 . 

21. A quotation from Robert Mager: "If we don't know where we are 
going, how will we know when we have arrived?" 

22. Behaviorally Stated Objectives ... Clarify Goals... 

23. What is a Behavioral Objective? This question is answered in 
Slide 24. 

24. A Definition of Behavioral Objectives and Slide 25. 

25. The Parts of a Behavioral Objective. Conditions, Performance, 
Extent, and the function of each part. 

26. How objectives and the systems-based approach differs from traditional 
approaches is answered, partially, in Slide 27. 

27. An Educational System. Further expalined in Slide 28. 

28. Showing the overall structure of an instructional system whereby 
the objectives are formulated. Directly after this formulation 

a criterion test is constructed, followed by the analysis of learning 
tasks, then by the design of the system itself. Evaluation is 
then made and finally the results of the evaluation are used to 
determine what changes, if any, are needed. This entire system makes 
possible a constant feedback of information dealing with success 
and/or failure of the total package, with some indication of where 
alterations, additions, or deletions are required. 

29. Slide 29 states that another point of variance from normal teaching 
situations is that this system is learner-oriented. That is to say, 
objectives are written from the learner's, and not the teacher's, 
point of view. Clearly stated objectives help eliminate what is 
shown in Slide 30. 

30. And perhaps your students are not seeing your objectives as you are. 
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NARRATION FOR OVERHEAD TRANSPARENCIES (continued) 



31. Slide 31 states that we use precision action words, not broad 
statements, in defining our goals. Precise words tend to clarify 
our goals, and, if they are action words, results can be measured. 

32. Slide 32 points out the fact that course goals and results are 
quite often two different things. 

33. This Slide shows many words certainly useful in goal statements 
which should not be used in behavioral ly or performance stated 
objectives. There are as many different interpretations of these 
words as there are people involved. 

34. Here are words that are more clear, more easily measured, and 
that are preferred over more general terms. 

35. This Slide points up the fact that all testing in this program 
is criterion-referenced evaluation and not norm- referenced 
evaluation in which we measure one student against another. 

36. Mastery teaching is emphasized over marking on a curve, normal 
practice in norm- referenced evaluation. 

37. Evaluation should pinpoint what we are doing right and what we 
are doing wrong. 

38. "Who gains from all this activity?" 

39. The student gains; 

40. The teacher is also a gainer; 

41. The administrator is a gainer; 

42. As well as the employer 

43. And the public. 

44. Slide 44 points up the way ESCOE intends to bring all of this 
about--through service to the LEAs. 

[Slides 45-53 point out various actions ESCOE is taking in its 
attempt to fulfill its stated goals.] 

45. Training facilitators from LEAs to write and edit behavlorally 
stated objectives. 

46. Conducting workshops in the LEAs to assist facilitators in 
training their faculties. 

47. Assisting in the development of objectives in selected areas of 
study through local facilitators, 

48. Codifying and placing these objectives in a computer data bank. 

49. Employing consultants in various subjesct areas to synthesize objectives. 

50. Developing the feedback loop by various devices such as simplified 
forms and check-off systems. 

51. Bringing to the Center subject area specialists to work with testing 
^ specialists in developing testing instruments. 
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NARRATION FOR OVERHEAD TRANSPARENCIES (continued) 



52, Training facilitators in test administration. 

53, Assisting in test administration at the LEAs. 

54, Slide 54 sums up with a statement that performance objectives 
"Say it like it isl" In other words, they clearly state what 
the student is to learn in a course or module of instruction. 

55, The Advantages. These stated advantages are but a small part of 
various advantages among many suggestions that have been given 
to the Center by its various participating LEAs. 

56, We can evaluate programs without standardizing them. 

57, It is really an excellent curriculum development tool. 

58, The entire program is always flexible, always open for modification. 

59, Slides 59 and 60 show a list of many other advantages possible 
through the implementation of the entire system,' 



For specification of this and other statements in the narration, write 

to Mr. Walter Long, ESCOE, Hudson Valley Community College, Troy, New York. 
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THE SYSTEMS APPROACH TO EDUCATION^ 



The ESCOE staff came to see behavioral objectives as components of a 
total systems approach to education. This list represents the rnodel 
taught to ESCOE facilitators. 



' 1 . STATE GOALS 



2. DEFINE OBJECTIVES 



3. CONSTRUCT CRITERION-REFERENCED TEST 



4. IDENTIFY I ND I V I DUAL LEARN I NG OBJ ECT I VES 



5. SELECT APPRORPIATE LEARNING STRATEGIES 



6. SELECT APPROPRIATE MATERIALS 



7. DEFINE AND ASSIGN APPROPRIATE PERSONNEL ROLES 



8. IMPLEMENT THE PROGRAM 



9. TEST AND EVALUATE STUDENT OUTCOME 



10. REF'NE AND REVISE AS NECESSARY 
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"HOW BEHAVIORAL OBJECTIVES ARE EVOLVED 
BY CURRICULUM DEVELOPERS OR TEACHERS" 

Extract from an Introductory Presentation 
Shown on Overhead Transparencies 

I. Approaches to Goal Analysis 

A. Goals. What are they? Where do they come from? 

Goals start out as statements like these: You should... 
It would be nice if,.. A citizen is... Education's 
responsibi I ity is. . • 

B. These broad educational goals come from: Students, Industry, 
Teachers, Home, Community, Nation,... 

The curriculum developer gets his goals from these sources 
and from using some approaches to goal analysis, such as: 

- Operationa li zat ion of Fuzzy Concepts' 

- Task Analysis 

- Critical Incidents Analysis 



- Operational ization. of Fuzzy Concepts 
(Hutch insonian method of goal analysis) 

Strategy for turning vague goal statements 
into measurable rerfor^mance statements 

- Task Analysis 

Analysis of the goal to determin^^ what the 
competent person does, or what performances 
are required, to demonstrate that the goal 
has been accomplished 

- Critical Incidents Analysis 

Determination of what was done (performance) 
when the Job was accomplished successfully. 

C- From the curriculum developer's expressional goals, performance 
statements in the form of measurable objectives are wHtten. 



Tom Hutchinson, The University of Massachusetts, School of Education, 
''The Operationalization of Fuzzy Concepts," 1971. 
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"HOW OBJECTIVE'S ARE EVOLVED" (continued) 
II. Some Categories of Objectives 



MEASURABLE 
BEHAVIORAL 
OBJECTIVES 



FALL INTO TWO CATEGORIES 



Pr imary 
I ntent 
Ob ject I ve 



I nd icator 
Objective 



An objective that measures 
the directly observable physical 
act that is_ the desired outcome 
of a program of instruction. 

A measure from which capabilities 
(cognitive and affective) may be 
inferred. 



Primary Intent Objective 

[The teacher thinks: "If I can observe the student coming 
down a slope on ski is and he makes it all the way to the 
bottom, then I will know that he can ski."] 

This objective operational i zed becomes the following 
primary intent objective: 

"Given skiis, poles, skiing clothes, and a snowy slope, the 

student will ski down the mountain without falling." 



Indicator Objective 

[The teacher thinks: "I would like to know if the student knows 
and can apply the law of levers."] 

This objective operational ized becomes the following indicator 
objective from which a capability may be inferred: 

"Given a bar 6 feet fn length, a small iron block to use as a 
fulcrum, and a 100-pound weight, the student will place the 
fulcrum under the bar at the proper position for a 50-pound 
force at the end of the lever to raise the weight," 
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OBJECTIVES: DEFINITIONS AND EXERCISES 



THE THREE COMPONENTS OF A BEHAVIORAL OBJECTIVE 



1 



Component 1 

PERFORMANCE 



Write a statement describing one of your educational 
intents and then modify it until it answers the 
question: "What is the learner doing when he is 
demonstrating that he has achieved the objective?" 



Component Z 

CONDITIONS 



- Define the desired behavior further by describing 
the important condition(s) needed to achieve the 
stated objective. 



Component 3 

EXTENT 



Add to this how well the student is expected to 
perform. 



EXERCISES 

On the following pages will be found examples of the exercises designed 
by the ESCOE staff and published in the Behavioral Objectives Training 
Package . This Training Package (for Facilitators and Teachers) became 
one of the major instructional instruments used by ESCOE and, later, by 
the facilitators in their workshops. 



'Operational definitions from an overhead transparency series produced 
by ESCOE. 
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OBJECTIVES: DEFINITIONS ANU EXERCISES (continued) 



EXERCISE 1: What Do You Think Behavioral Objectives Are? 
(You may check more than one answer.) 

A. Statements that give the instructor a general outline of the 

course structure. 

;-'BV Statements that tell the instructor exactly what the students 

are to learn, how well they are to learn, and any necessary 
conditions under which the learning is to take place. 

C..A series of understandings for the students to learn in class. 

D. The behaviors the student should acquire during the course of 

instruction. 

Answers: B, D 

EXERCISE 2: What Forms A Behavioral Objective 

Identify the three components of a behavioral ob^ '.ive by writing 

in the space provided next to each statement whr' r it is a Condition, 

Performance, or Extent. 

1. Engineering Related Technology 

A. The student will trace the light path from its source by 

naming the parts through which it must pass. 

B. At least 4 out of 5 parts correct 

C. Given a line diagram of an optical metallurgical microscope 

2. Auto Mechanics 

A. %% accuracy, time 15 minutes 

B. List in writing the properties in a coolant 

C. From memory without reference material 

3. Agricultural Technology 

A. The student will draw the cooling curve for the given alloy 
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EXERCISE 2 (continued) 



B, A complex schematic or actual equilibrium diagram and a 
specific alloy composition 

80% correct 



Answers: 1. A, Performance 2, A, Extent 3. A, Performance 

B, Extent Performance B, Conditions 

C. Conditions C. Conditions Extent 

4, Write a brief description of each component: Conditions, Performance, 
Extent. 

5. List a few brief examples of each component. 

EXERCISE 3: Are You Writing Good Behavioral Objectives? 
Pause to see if you can check the correct answers. 

1. Which of the following words would be used in a good behavioral objective? 

A. Grasp C, List ^E. Compute G. Understand 

B. Believe ^D. Measure F. Identify H. Know 

Answers: C, D, E, F 

2. Which of the following is NOT a behavioral objective component? 

A. Statement of desired behavior (what the student is expected to 

do to demonstrate achievement of the objective). ^ 

B. Delineation of the teacher's role in the learning process. 

C. Specification of conditions under which behavior should be 

demonstrated. 

D. Statement of the criteria for success. 

Answers: B 
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EXERCISES (continued) 



Exercise 4: A Good Behavioral Objective: 

^A. Is stated in directly observable performance terms 

^B. Allows for considerable ambiguity (can be interpreted differently 

by different people). 

^C. Can be used as a yardstick to assess a student^s degree of 

achievement: how well he has accomplished what he was expected to do 

^D. Is stated with sufficient specificity and clarity to be useful 

E. Is a highly complex development 

^F. Talks about general goals of learning 

Answers: A, C, D 

EXERCISE 5: How could the following statement be converted into 
behavioral terms: "Appreciates the value of the 
scientific method." 

A* Knows the scientific method and applies it effectively 

B. Understands the principles of the scientific method 

^C. Uses correct experimental procedures in problem solving 

^D. Is able to list the basic principles of scientific procedures. 

Answer.: D 
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ESCOE CONFERENCES 



Eight major conferences for the purposes of planning, training, and 
dissemination for ESCOE administrators and facilitators have been 
conducted by The Evaluation Service Center since its inception. 
These conferences are described on the following pages. 



Administrators* Conferences: Rationale 

It has been a point of major concern that administrators (principals, 
directors, superintendents, etc.) of LEAs, although they might not be 
directly involved in the writing of objectives, be fully supportive 
of the members of their staffs facilitators and teachers who 
were directly involved. Conferences focusing on planning and report- 
ing of ESCOE activities and services were emphasized for the 
administrators . 



Facilitators' Conferences: Rationale 

The facilitators' role in the project has been: 

i 

I 1. organization and coordination of the process of preparing behav- 
j ioral objectives in each LEA; 

I 2. liaison between the school administration and faculty to expedite 
preparation of objectives, including appropriate allocation of 
staff tine; 

3. editing of all objectives in preparation for transmittal through 
the Center's field representatives to ESCOE; 

4. impressing both administrators anci teachers with the value to be 
i received from use of the data bank and related services from 

i ESCOE. 

i 

i 

; Conferences for facilitators have been directed towards preparing 
! this group for their responsibilities in the LEAs and to ESCOE. 
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CONFERENCES (continued) 



CONFERENCE I 

Initial Administrators* Planning Conference 
September 1970 
Albany, New York 

Approximately 25 administrators from 10 LEAs in Massachusetts 
and 10 in New York met with project developers for planning 
and to make a commitment to the project. 



CONFERENCE II 

First Facilitators* Training Conference 
November 1970 
Amherst, Massachusetts 

Thirty-five facilitators from committed LEAs (10 in each state) 
attended this conference to acquire skills in writing behavioral 
objectives. At this conference the format for submission of 
objectives to ESCOE was developed. 



CONFERENCE III 

Second Facilitators' Training Conference 

January 1971 

Amherst, Massachusetts 

Thirty-five facilitators, from 10 LEAs in each state, attended 
this follow-up conference to consolidate skills previously 
acquired and to plan for the coming year. 



CONFERENCE IV 

Dissemination and Planning Conference for All Facilitators 
May 1971 

Bass River, Cape Cod, Massachusetts 

Thirty-six facilitators from 10 LEAs in Massachusetts and 8 in 
New York attended this conference for the dissemination of raw 
objectives in the form of printouts from the data bank. Instruction 
was given in how to read and use the printouts and in the use of 
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CONFERENCES (continued) 



the data bank for program analysis and development. Information 
on the selection of objectives and on the extension of the data 
base to more program areas was given, as well as on test develop- 
ment. Plans for the 1971-72 school year were discussed. 



CONFERENCE V 

Administrators* Planning Conference 
September 1971 
New York City 

Thirty-five administrators from 13 Massachusetts LEAs and 9 New 
York LEAs attended the conference. Six administrators from 5 
Pennsylvania schools also attended. The purpose of this conference 
was to develop a detailed agreement between the LEAs and The 
Evaluation Service Center. This agreement was specific for both 
parties and described the services the schools could expect from 
ESCOE during the year and the expectations that ESCOE had of the 
LEAs. (This Service Commitment 1s reproduced elsewhere in the 
Final Report,) The conference also presented a comprehensive des- 
cription of ESCOE services and a demonstration of the work already 
developed. 



. CONFERENCE VI 

Second-Year Facilitators' Conference 

October 1971 

Chicopee, Massachusetts 

Fifteen facilitators from 7 Massachusetts LEAs and 5 New York LEAs 
attended the conference. The purpose of this conference was to review 
ESCOE and LEA activities of the past school year, to teach participants 
about the process of synthesis and the uses of synthesized objectives 
that were developed over the summer, and to resolve plans for the 
present school year. The major portion of Instructional time was spent 
on synthesis; addresses on the systems approach to program improvement 
and on computer usage were also given. 
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[continuei on next page] 



CONFERENCES (continued) 



CONFERENCE VII 

Training Conference for New Facilitators 
November 1971 
Chicopee, Massachusetts 

Thirty-five facilitators from 7 LEAs in Massachusetts and 11 LEAs 
in New York attended- This four-day conference concentrated on 
training facilitators new to the project in the content and techniques 
necessary to their fulfilling their commitment to their LEAs and to 
ESCOE. The agenda reproduced below will give a good idea of what 
such training includes. 



AGENDA 

"A Systems Approach to Program Improvement" 

"Introduction to Behavioral Objectives" 

Writing Behavioral Objectives - Small Group Exercises 

Exercise in Editing Objectives - Small Groups 

"A Glimpse at Domains" [Psychomotor, Cognitive, and Affective 

Classifications of Tasks] 
Behavioral Objective Reporting Forms - Explanation and Exercises 
Test Development: Principles and Status 
"Synthesis of Objectives: A Brief Overview" 
"The SYNOB Package: An Introduction^' 

Development and Selection of Objectives - Explanation and Exercises 
Implementation and Usage of Objectives - " " " 

Plans for the Future: Discussion 



CONFERENCE VIII 

Spring Dissemination and End-of-Project Conference 

for Al 1 Faci 1 i tators 
May 1972 

Monti cello. New York 

Thirty facilitators from 8 Massachusetts LEAs and 12 New York LEAs 
attended this final ESCOE conference- Questionnaires for the projtxt 
evaluation were administered and a comprehensive review of the project's 
activities undertaken- The future of the project's developmental v^ork 
in Massachusetts and New York was presented. An overview of the 
principles and activities of the Comprehensive Achievement Monitoring 
(CAM) services was given by Dr. William Gorth and a representative from 
the New York State Department of Education spoke on the Special 
Education Instructional Materials Center (SEIMAC). 
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COURSE CREDIT FOR ESCOE FACILITATORS 
through The University of Massachusetts 
Division of Continuing Education 



"The University of Massachusetts 
is offering credit, through the 
Division of Continuing Education, 
for each year of active partici- 
pation as a facilitator in the 



* Systems Management for 
Occupational Education, ' 
1971. 5 credits. 



14 Facilitators enrolled 
in Education 386/686, 



ESCOE project." 



from a memo to all facilitators 
November 1971 



41 Facilitators enrolled 
in Education 383/686, 



"The purpose of the course(s) is 
to allow teachers in occupational 
education to earn University 
credits by direct involvement with 
a research project. The ESCOE 
project is philosophically rooted 
in teacher-generated input." 



'Systematic Approach to 
Curriculum Development 1n 
Occupational Education. ^ 
1972. 5 creditbs 



Fees were the 



usual $10 and $15 
per credit for 



from a memo to all facilitators 
April 18, 1972 



Massachusetts and 
out-of-state resi- 
dents respectively 



COURSE DESCRIPTIONS 

1971/Education 386/686: 'Special Problems in Education: Systems 
Management for Occupational Education.* Instructor: Alfred R. Rios, 
Assistant Director, Evaluation Service Center. 5 credits. 



Students will attend two workshops each semester, to be 
conducted by the ESCOE staff, and will participate in one 
meeting per week within their own LEAs. A two-hour per 
week work schedule under the direction of the ESCOE staff 
will include: 

1. A study of developing a systems approach to 
student learning environments; 

2. implementation of a model for developing 
behavioral objecti ves ; 

3. training of teachers in the writing of 
behavioral objecti ves ; 

4. writing objectives for student's own course 
of study (Education 386/686). 



[14 facilitators completed this course.] 
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COURSE CREDIT FOR ESCOE FACILITATORS (continued) 



1971 /Education 386/686: 'Special Problems in Education: Systematic 
Approach to Curriculum Development in Occupational Education/ 
Instructor: Kenneth Ertel , Director, Center for Occupational Education. 
5 credits. 

Facilitators will attend the three ESCOE conferences 
planned for this year. They will conduct workshop^ in 
their own schools under the direction of the ESCOL field 
staff. They will train teachers to write behavioral 
objectives according to the ESCOE format. Facilitators 
are responsible for the orientation cf teachers in their 
schools as to use and implementation of objectives. 

Facilitators' training will be aimed at their 
learning to: 

1. develop a systems approach for learning environ- 
ment; 

2. write and use behavioral objectives for local 
programs ; 

3. train teachers and conduct workshops in behavioral 
objectives ; 

4. synthesize objectives for an occupational program; 

5. use criterion-referenced tests to evaluate student 
achievement; 

6. modify curriculum based on system feedback. 
Suggested readings: 

1. Developing Vocational Instruction . Robert Mager. 

2. Writing Performance Goals: Strategy and Prototypes . 
McGraw-Hill . 

3 . Instructional Systems Development for Vocational and 
Technical Training . F. Coit Butler. Educational 
Technology Publications. 

[41 facilitators completed this course.] 
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IV 



SYNTHESIZED OBJECTIVES 
CONCEPT, PRODUCT, AND PROCESS 



SYNTHESIZED OBJECTIVES 
THE CONCEPT AND PRODUCT OF SYNTHESIS 



THE CONCEPT 



AUGUST 1971 



behavioral objectiv/es that was developed by 



ESCOE has expanded a model for synthesizing 



2571 RAWOBs in 
4 subgroups 
synthesized into 
343 SYNOBs, 



David Berliner in 1969.* The synthesized 



JUNE 15, 1972 



objective provides a sophisticated form that 



1354 RAWOBs in 
12 subgroups 
synthesized into 
381 SYNOBs. 



is more efficient than individual objectives 



and allows for a variety of local instruc- 



construction also required a consolidation 



tional needs and preferences. Test 



1667 RAWOBs in 
10 subgroups 
"in process'* of 
synthesis. 



of objectives according to similarity of 

performance. Identical or similar performances from different LEAs 
were written as one objective with a variety of "form changes." 
Substitution of a form change for a portion of the fixed text 
yields equivalent forms of the same task. In a well-written SYNOB 
the "performance" is fixed and without form changes; form changes 
will be found only in the "'conditions" and "extent" portion of the 
SYNOB. 

THE PRODUCT 

Synthesis of objectives in four subgroups (Automotive Mechanics, 
Cabinetmaking & Millwork, Industrial Electronics, and Machine Shop) 
was begun in July of 1971 and completed in August 1971. A total of 
2571 raw objectives in these^ subgroups were synthesized into 343 

*See pages 60-63 
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SYNTHESIZED OBJECTIVES: THE PRODUCT (continued) 



SYNOBs. In October of 1971 these SYNOBS were entered into the data 
bank. 

LEAs received computer printouts of SYNOBS in the above four sub- 
groups and were asked to indicate which portions of their programs 
were covered by the SYNOBs and form changes for those subgroups. 
This request was made by ESCOE in order that test development 
personnel would be able to recognize the extent and limitations of 
the information with which they were working. 

In March of 1972 nineteen facilitators and teachers began synthesizing 
objectives in ten occupational areas comprised of twe^'uy-two sub-- 
groups. As of June 15, 1972, synthesis for twelve subgroups was 
completed, with a total of 381 SYNOBs derived from 1354 RAWOBs. 
Synthesis for the remaining ten subgroups will be completed in July 
1972. At that time all SYNOBs will be entered into the data banks 
of both states. The ten occupational areas for which objectives have 
been synthesized during school year 1971-72 are the following: Auto 
Body, Baker, Chef/uook, Commercial Art Occupations, Drafting Occupa- 
tions, Electrical Occupations, Graphic Arts Occupations, Metal 
Fabrication, Practical Nursing, and Woodworking--all at the secondary 
level . 

Examples of four synthesized objectives, exactly as they appear on 
the computer printouts, are given on the two following pages. 
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SYNTHESIZED OBJECTIVES 
THE PROCESS OF SYNTHESIS 



"Synthesis": ESCOE Definition 

The process of combining individual, indepen- 
dently generated raw objectives for the same 
performance into a single, "synthesized" 
objective— or '^SYNOB." 



The first step in the process of synthesis is to retrieve objectives 
from the data bank, one objective per page, by Block and Unit classi- 
fi cations within Subgroups. The raw objectives are then manually 
sorted according to similarity of performance. 



THAT IS TO SAY: 1. collect RAWOBs in subgroups 

2. code RAWOBs by Block and Unit 

3. group RAWOBs within the Units by 
similarity of performance 

4. look for similarities of performance 
that may exist across Units and group 
RAWOBs further. 

The groups of raw objectives are now ready for synthesis. 



MECHANICS OF SYNTHESIS 

For objectives with similar PERFORMANCES, make three li~sts, one for 
each component of the objectives: Performance, Conditions, Extent. 
From these lists, first remove the redundancies, then identify the 
fixed (most typical) text, and the variable (atypical) text, which 
become the "form changes." Then write the synthesized objective. 
The process of synthesis can be seen in schematic form on the 
followinq pages. 



SYNTHESIZERS IN THE LEAs 

Synthesis for each subgroup was accomplished by two teachers and/or 
facilitators working in the instructional areas of their expertise. 
Special training workshops in the process of synthesis were held in 
the LEAs, in addition to the emphasis on synthesis at the Conference 
for Second«year Facilitators in October 1971. ESCOE staff was avail- 
able for consultation at LEAs and continued to conduct smaller work- 
shops in synthesis throughout the year. 
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SYNTHESIZED OBJECTIVES: THE PROCESS — Schematic 

Conditions, Performances, Extents for several objectives combine 
into one objective with form changes as options. 



ERIC 



SORT RAWOBs 
[by Blocks & Units] 



PERFORMANCE 



CONDITIONS 



EXTENT 



PERFORMANCE 



CONDITIONS 



EXTENT 



PERFORMANCE 



CONDITIONS 



EXTENT 



COMPILE 3 LISTS 



PERFORMANCES 



CONDITIONS 



EXTENTS 
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WRITE SYNOB 
[eliminate redun- 
dancies ; identify 
fixed & variable 
text] 



PERFORMANCE 



CONDITIONS 



i.n 



1.12 



EXTENT 



3.11 



3.12 



OVERVIEW OF THE PERFORMANCE CRITERIA IN 
MACHINE SHOP EDUCATION^ 



The decision was made to try and systematize the performance criteria 
in one area of vocational education. The area chosen was machine 
shop because it is a standard subject area in most vocational educa- 
tional programs. Moreover, all schools contributing input to the 
vocational education evaluation system wrote behavioral objectives in 
this area. Further, since a system for codifying behavioral objec- 
tives was needed, a single representative e^ea had to be chosen for 
pilot examination. 



The first task undertaken was the development of a coding system for 
retrieval of behavioral objectives by domain. A system was devised 
that with minimum modification could be used in a computer retrieval 
system. The code used is as follows: 



2 
2 
3 
2 
2 
1 



digits - area or course designation 
digits - division within course 
digits - unit 

digits - unique identification 
digits - school code 
digit - grade level 



Thus a string of digits such as 01 01 002 01 05 2 would signify 
machine shop as the course (01), lathe work as the division (01), 
straight turning as the unit within lathe operation (002), and that 
this IS the first objective under that subclassification, i.e., its 
unique identifier is (01). Further, the (05) signifies that this 
behavioral objective is taught at Haverhill, and the final (2) 
indicates its occurrence is at the 10th grade. A small part of the 
system developed for machine shop is illustrated below. 



rse Division Unit Unique 


Course 


Division 


Unit 


Unique 


Shop 


01 








Lethe 


01 


01 






Threading-Internal 


01 


01 


001 


01 


-External 


01 


01 


001 


02 


Straight Turning 


01 


01 


002 


01 


Tapering 


01 


01 


003 


01 


Bori ng 


01 


01 


004 


01 


Reaming-straight 


01 


01 


005 


01 


-tapered 


01 


01 


005 


02 


Necking 


01 


01 


006 


01 


Chamfers & Aryles 


01 


01 


007 


01 


Set-up 


01 


01 


008 


01 


Drill Press 


01 


02 







etc. 



^David Berliner, University of Massachusetts, 1969. This is the origi- 
nal conceptual statement on which the process of synthesis at ESCOE was 
based. 
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OVERVIEW OF PERFORMANCE CRITERIA (continued) 



The second major task undertaken was the unification of the behavioral 
objectives received from various sources. A single statement of a 
behavioral objective was written, incorporating the common elements 
intrinsic in the various statements of performance for a given machine 
shop task. Thus the five or six statements on lathe work with use of 
a holding device were analyzed and a single statement of objectives 
was written. To satisfy each of the local schools, whose autonomy 
was in no way to be impuned, we have developed a system of form 
changes for the behavioral objective which gives wide lattitude in 
usage for each school. (This is in addition to a retrieval system 
that allows each school to have, should it desire it, its original 
statement of performance.) Thus, with this approach we might have 
taken the following behavioral statements: 

(A) When given a blueprint of a job containing an angle, 
the student will produce the part using a compound 
rest , adhering to indicated tolerances. Tolerance at 
this level should not be less than plus or minus 1/2". 

(B) When given a blueprint of a job containing a chamfer, 
the student will produce the part using the compound 
rest, adhering to indicated tolerances. Tolerance at 
this level should not be less than plus or minus 1/64"; 



synthesized them and developed this single revised statement: 

Given a (1) blueprint of a job containing an (2) angle 
the student will produce the part using a compound 
rest, adhering to (3) tolerance of +1/64". 



We could develop from the behavioral objectives the following form 
changes. (Those aspects of the single revised statement which can 
be modified were underlined and numbered above. Substitutions for 
these key words are then provided.) 

(1) piece of stock; model to match 

(2) chamfer 

(3) +1/2".- 



The three substitutions, all roughly equivalent forms of the saine 
behavioral task, though perhaps requiring different levels of coiiipe- 
tency and experience to perform them, provide the teacher with 3x2x2 
different behavioral statements, >ach of which taps the skill that is 
important to the teacher. In some cases this technique of specifying 
form changes yields an infinite number of roughly equivalent. 
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OVERVIEW OF PERFORMANCE CRITERIA (continued) 



i nominally parallel, forms of the performance test a student is to 
take. The level of difficulty of a task, perhaps best noted in the 
specification of tolerances, can be manipulated according to grade 
level since different schools give instruction in different skills at 
various points in the curriculum. Further, tasks can be done early 
in a program at one level of difficulty (e.g., tolerance required is 
±1/32") and after other skills develop (i.e., ability to use a micro- 
meter) a different level of tolerance may be set (e.g., tolerance set 
at +.001). 

Thus the form changes are seen to be an important new development in 
the specifying of behavioral objectives. The characteristic of the 
form change which allows for different levels of competency to be 
demonstrated in the accomplishment of a particular performance points 
toward that aspect of the system called screening tests. For 
different levels of performance different prerequisite skills are 
required. In the example cited above one level of competent perform- 
ance could only be achieved with prerequisite knowledge of a ruler. 
To achieve the higher level of competency prerequisite skills must be 
mastered for using the micrometer. The screening tests then serve 
the instructor with a handy guide for determinging at least some of 
the knowledge and skills necessary for a student to have mastered 
before attempting a machine performance. Both knowledge and skills 
tests can be described in that section called screening tests. 

Most of the objectives received were written with machine or hand 
skill performance in mind. The related science aspects have not, at 
this time, been effectively described. In the section provided for 
screening tests associated with each behavioral objective those as- 
pects of a related science which are prerequisites can be specified. 
Thus at the simple level, computation of Ohm's Law is seen as a 
necessary prerequisite for circuit testing in electronics; knowledge 
of the function of twelve important parts of a lathe might be a 
screening test for certain lathe operations; load-stress and dura- 
bility information about different materials may be important prereq- 
uisite knowledge about upholstering operations; recognition of each 
kind of cutting tool may be prerequisite to certain cutting machine 
operations, etc. 

Many paper and pencil tests can be developed for inclusion in the 
category of screening tests. Simulated techniques, utilizing scale 
models of machines may be developed and used to screen students. The 
major point is that the use of screening tests completes the descrip- 
tion of specifications about a task because they specify the level of 
the entering behaviors a student must have mastered before one wants 
to allow the student to try a skilled performance on a machine. Time 
IS too precious, and the costs of equipment too great to merely allow 
anyone who wishes to attempt an objective free reign. The screening 
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OVERVIEW OF PERFORMANCE CRITERIA (continued) 



tests, after appropriate psychometric study, should contribute much 
to the practical working of the vocational education system. 

It should be noted that the collection of many behavioral objectives, 
form changes providing many "equivalent forms" for performance test- 
ing, and screening tests, constitutes an example of curriculum 
development which is of tremendous utility. The entire machine shop 
curricula for the state can be described, schools and districts can 
obtain comparative information, and can choose and pick those objec- 
tives that should be taught or than can be taught in their own geogra- 
phic areas. Moreover, evaluation for each objective will provide an 
overall picture of vocational education within the state, with no 
sacrifice to local autonomy since the form changes allow for diversity 
of goals. The concept of readiness or entering behaviors in curricu 
lar formulation is treated in fact, and not just given lip service, 
through the use of the screening tests. It is believed that this 
model of curriculum development, which incorporates diversity, avoids 
all the old fantasies about centralized, mechanized, dnd systematized 
imposition of goals on local districts and, at the same time, provides 
a framework within which centralized, mechanized and systematic eval- 
uation of goals can be undertaken. 



-63- 



THE FEEDBACK LOOP 
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INTRODUCTION TO V: THE FEEDBACK LOOP 



The basic feedback process consists of the processes described below. 
LEAs send raw objectives to ESCOE on the Behavioral Objective 
Reporting Form (#12B). ESCOE returns to the LEAs computer printouts 
of the raw objectives and also a matrix showing the total number of 
raw objectives submitted to ESCOE by each LEA in each Subgroup, in 
each Block within the Subgroup, and in each Unit within the Blocks.. 
When compared with the Block and Unit Breakdown for a particular Sub- 
group, the RAWOB Matrix informs each LEA where its own contribution 
of behavioral objectives fits into the ESCOE data bank. Then, when 
objectives for a Subgroup have been synthesized, printouts of the 
SYNOBS are sent to each LEA, along with a SYNOB Matrix which, as with 
the RAWOB Matrix, indexes the synthesized objectives within a Sub- 
group by Blocks and Units. With this information, the LEAs return 
to ESCOE information on the appropriateness of the SYNOBS in question 
for the occupational areas offered by an LEA. Such response is re- 
turned to ESCOE via the SYNOB Selection Reporting Form {Form A) and 
the Block and Unit Coverage by SYNOBs (Form B). Those forms bearing 
j feedback from the LEAs to ESCCE are the subject of this section. 

I Further details may be found in the Instruction Manual: Synthesized 

i * 
i Objective Package , 
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OBJECTIVE REPORTING FORMS 



I BEHAVIORAL OBJECTIVE REPORTING FORM (#12B) 

i 

■ This fom is the one on which raw objectives have been submitted bv 

I 

j the LlAs to ESCOE. Besides the objective itself, the Behavioral 

I Oojective Reporting Form asked for information identifying that objec- 

tive in many dimensions, which then could be used to retrieve the 
; objective from the data bank. The particular layout of this form may 
well not be useful in a context other than that of ESCOE; however, an 
acquaintance with the kinds of information requested about an objec- 
tive may p.^ov? useful to others undertaking a similar project. A 
full explanation of Form #128 follows the sample of this fonn. 

OBJECTIVE EDITORIAL FORMS 

Form D and the Objective Editorial Comments form were each used by 
ESCOE for response from the LEAs to objectives on the RAWOB printouts. 
A response on Form D might include "second thoughts" on the writing 
of an objective, its phrasing or emphasis, as well as factual correc- 
tions. These tv/o forms are reproduced at the end of this section. 
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BEHAVIORAL OBJECTIVE REPORTING FORM (continued) 



CODING HEADER 



It) Number 




T 


N 


C 


SI. 


Ci t:y-Schoo1 


1 LvL 1 


No. 


ik. 






































1 1 1 







Cap. C). F] , 


OR J 


SG 




MT 


U 




Related Sub.i. DUcioline 






































1 1 









I.D. No. 



Yr, 



T.N.C. 



The number assigned by the clerical staff at ESCOE 
to identify each objective, according to when it 
was received at ESCOE. Thus, if a particular LEA 
submitted 100 objectives on a certain day, and there 
are already 8000 objectives in the data bank, the 
objectives being submitted will carry the identification 
numbers 008001-008100. 

Year. The year in which the behavioral objective 
was written: 70, 71, or 72. Also to be written in 
the space provided below the coding header. This 
information was requested on the assumption that it 
will be necessary for future output concerning 
popularity and longevity of the objective. 



Type Numbers Column, 
staff. 



Coding ror use of the key-punching 



St. 

City-School 
Lvl. 



No. Tk.' 



State. The name of the state in which the participating 
LEA was located; each state was assigned a code number. 
Also to be written in below coding header. 

The USOE code number assigned to each LEA. 
Name of LEA to be written in uelow header. 

Level, A two-digit code number identifying the type 
of program the LEA provides and the leve^l of the program 
at which the objective is offered. The fir^t digit 
represents the program level and length and the second 
the ^ear within the proaram when the objective is offered. 
See page 70 of tnis report for the list of programs 
represented and their code numbers. Also below coding header. 

Number Taking, The number of students to which the 

objective is (will be) taught during the school year. 

To be written in space below coding header as well. ^ 
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[continued on next page] 



BEHAVIORAL OBJECTIVE REPORTING FORM: CODING HEADER (continued) 

ft 



Cap. CI. Capability Classifi cati-^n. An in-house codiig scher^e 

to identify which of the items under this head on the 
reverse side of 12B have been checked > for example: 

[X] Psychomotor 1 
[ ] Psychomccor 0 

fx] C 1.1 Knowledge of Specifics — 1 1 
etc, \ 
See page 71 of this report for more 'detail on Capability 
Classificati ons. 



^ 



Fl ^ Field. T^^e USOE codes for different occupational 
GR Major Group. ai^eas, for example: 
SG Subgroup. 

Field of Study: Trade & Industry U.0OOO- 
Major iiroup: Automotive 

Industries 1703^00 

Subgroup: Auto Mechanics • 1 70302 

This information is to be written in (in words) below 
the coding header as well. 



BL ^ Block. 

KT Major Topic (Post-secondary only)** 

UN Unit. 

The coding scheme developed by ESCOE to identify the 
instructional segments within a subgroup^ For instance, 
within the Subgroup Auto Mechanics 170302 the foll\)wing 
BlocL is one of many in the Subgroup, each with its own 
Units: 

^ Block 01 Pov^er Transmission 

Unit 01 Engine 

02 Transmission, Standard 
, 03 Transmission, Automatic 

Blocks and Units were disseni nated to LFAs as they became 
aVfjilable. See the Appendicc^ of this report for further 
information on Blocks and Units. (This information to be 
0 written in below coding header;) 

**For' objectives written by Post-secondary LEAs, the 
Block is the name of the course; the Major Topic is 
the major breakdowns of the course; the Unit shall be 
the units of instruction below the Major Topic. Block 
and Unit Breakdowns, with Major Topics, were constructed 
separately from secondary Blocks and Units. 
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BEHAVIORAL OBJECTIVE REPORTING FORM (continued) 

Related Code numbers developed by ESCOE to identify which of the 

Subject subjects listed on the reverse of 12B were checked as being 

Discipline "related" to the objective being submitted. The usefulness 

of such information may be seen, for example, in that if 
objectives are coded by Natural Science discipline, it is 
possible to determine the common mathematics capabilities 
sought by all occupational program? within or across LEAs. 



SPACE BELOW CODING HEADER 

In addition to the information to be written out as well as coded 
in the header, the names of the facilitator in the LEA and the 
instructor who wrote the objective are requested so that they may 
be consulted if necessary during the editing process. This information 
is not entered into the data bank. 



CODES FOR "LEVEL" 
Program Year 

Level Offered Type of Program Represented 

Pre-(/ocational , including exploratory 
Secondary, 1-year program 
Secondary, 1st year of 2'-year program 
Secondary, 2nd year of 2-year program 
Secondary, 1st year of 3-year program 
Secondary, 2nd year of 3-year program 
Seccndary, 3rd year of 3-y^ar program 
Secondary, 1st year of 4-year program 
Second^ary, 2nd year of 4-year program . 
Secondary 5 3rd year of 4-year program 
Secondary, 4th year of 4-year program 

Pre-post-secondary program 

Post-se^condary , 1st year of 1-year program 
Post-S'icondary , 1st year of 2-year program 
Post-secondary, 2nd year of 2-year program 

Other 



0 


0 


1 


1 


2 


1 


2 


2 


3 


1 


3 


2 


3 


3 


4 


1 


4 


2 


4 


3 


4 


4 


5 


1 


6 


1 


7 


} 


7 


2 


8 


1 
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BEHAVIORAL OBJECTIVE REPORTING FORM (continued) 



CAPABILITY CLASSIFICATION 

Fom .l^B - Reverse CAPABI LITY CLASSir ICAH ON ^^^^^J^^^^ 

I Psych^notor ""''"'''"^ 

I [ ] Psychomotor (Check only if performance requires significant 

; musr^lar actwUy} 

' Cognitive 

I (Check be one cognitive capability that best describes the ment,*! n 

' activity fnvoJved) 

j Knowledge Acquisition 

f [Id.) KnowledgL- of Specifics 

■ [ i CI.? Knowledge o-^ Ways and Means of Dealing with Specifics 

Knowledgp Application 

[ J CZ.l Knowledge Application Without Alteration 
[ J C2,2 Knowledge Appltcatlon With Alteration 

Introducti on 

The performance of a behavioral objective infers or denotes certain 
abilities the': may be classified, broadly speaking as "psychomotor," ; 
"cognitive," "affe'-tive," or combinations thereof. Psychomotor 
capabilities are those that are mostly muscular in nature although 
ensuing from cognitive capabilities. In general, psychomotor 
capabilities involve manipulating objects with various parts of the 
body. Cognitive capabilities are those that are pri(7?arily intellec- 
tual or mental in nature. In general, these capabilities involve 
acquiring and applying knowledge or information. Affective capabili- 
ties are those chat are mostly^einotional in nature. ^ They generally 
involve acquiring or manifesting a feeling or attitude towards a 
particular object, person, or idea. 



For further elaboration of the Capability Classification Scheme, see 
pages 62-69 of The behavioral Objective Training Package . 
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SYNTHESIZED OBJECTIVE FORM (#11 A) 



The Synthesized Objective Form is that form used by facilitators 
and teachers performing synthesis for ESCOE to record the SYNOB 
itJ^elf and varied Information about the raw objectives from which 
the SYNOB was derived, as well as the numbers of the Blocks and 
Units upon which the SYNOB touches. In this way, each SYNOB is 
"cross-referenced," so to speak, with many relevant pieces of 
inforniaLion. The instruction sheet for using the form, along with 
an 11 A referenced to it by numbers, as well as a blank 11 A, is 
included in this section- 
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SYNTHESIZED OBJECTIVE REPORTING FORMS 



The Synthesized Objective Selection Reporting Form (Form A) and the 
Block and Unit Coverage by Synthesized Objectives (Form B) comprise 
the feedback loop from the LEAs to ESCOE, in response to the SYNOB 
Printouts and Matrices. Fcnns A and B appear on the same page in this 
section. 



FORM A: SYNTHESIZED OBJECTIVE SELECTIG., REPORTING FORM 
Form A provides an opportunity to indicate, for eve . SYNOB applicable 
to programs in that I FA, v^iich form changeCs), if any, it uses by 
inserting the apprcpriate form change number (found on the SYNOB print- 
out). The LEA may indicate the need for additional form changes with- 
in a component (rondition, Performance, or Extent) of the SYNOB by 
simply writing in the space provided the additioiial change and desig- 
nating it by the next highest form change rumber for that component. 
There is also space to make general comments on either the separate 
components or on the synthesized objective as a whole. For example, 
an LEA may disagree with a portion of the fixed test and indicate 
this in the place provided by inserting its own version. A sample 
0-^ a completed Form A appears on the next page. 
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1 FORM B:. BLOCK AND UNIT COVERAGE BY SYNTHESIZED OBJECTIVES 

t 

I The SYNOB Matrix locates SYNOBS, by ID numbers, that "touch upon" 
j specific Blocks and Units. Form B asks each LEA to indicate which 
I portion of its course of instruction in the specified Subgroup still 

' remains to\be covered by SYNOBs. Thus a percentage figure in a part- 

\ 

I icular square indicates what portion of this LEAs course (by Blocks 

' and Units) has not been covered by SYNOBs. An X in a square means 

I 

; that unit of instruction is adequately covered by the ESCOE SYNOBs. 
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SYNTHESIZED OBJECTIVE FEEDBACK FORMS (continued) 
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SYNTHESIZED OBJECTIVE SELECTION REPORTII'-G FORM (A) - COMPLETED 5AMPLE 



Form A 



School 
SubgrOsip 



Instructor 
Facilitator 



EVALIMION SERVICE CENTER FOR OCCUPATTONAL EDUCATION 



?ypJ^Jl?.i^jLPi^J ective Selection Reporting Form 



October 1971 



170302 



Da Ui 



SYNOB ID f> 170302/001 

Level 21 

No, of Pup1^s25_ 



1*0 CONDITIONS 



2*0 PERFORMANCE 



3*0 EXTENT 



r ] 

[ 1 

H 

C ] 

[]—-= 



GENERAL COMMENTS: 

How well does this Synthesized Objective cover 
your course of Instruction? 

[ ] % 

2.32 - Do not re move fit & replace 
wrist pins, R & R Connecting Rods 
& Install. 



Wri">t pins are better called Piston 

pins. 

2.42 - Knurl should read Knurl 

Piston Skirts . 



The synthesized objective to which tin's particular Fom A refers 
may be seen on page 14 of the Instruc tion Manual: SvnthP.n^pH 
Objective Pa ckage, a separately bound appendix to this report. 
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FORMS A ?i B 



Form A 



School 

Subgroup 

Instructor 

Facilitator 



rTO. of PuDils 



October 1971 



Da:. 



1»0 CONDITIONS r ] 

[ J 
r "i 

r i 
[ ] 



2*0 PERFORt-lANCE 



3^0 EXTENT 



I], 



u 

[ ] 
[ ] 
[ ] 



.'U f.o:-y,F:r''TS'. 

'i'^,. w;;ll lioc: this Syr.thpsi ?.ed Objective cover 
:<A:r-:s of instructi'jn? 
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Form B 



School 



EVALUATION SEf^VICt CZNTlk iOk 'JCCUPATIOf^/VL EOUCATIOH 

LE A RCPORT I ! .;_G_ FORK 
Block and Unit Co vera gf; by Synth»=si.-'.oc; Objectives 

Bate 



October 1971 



Subgroup 



UNITS 



01 
02 
03 
04 
05 
06 
07 
08 
09 

10 
11 
12 
13 
14 
15 
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!„_ J . 


i 


































\ 














! 
i 




i 






































1 










1 1 ! 


L__ 








































1 


































































1 i 

! : 
























































1 

--t-- 
























































— - ■ 






















































t- — 
i 




























































- 
























































[. . . 



























































1 
t 
























































1 
i 




i 
1 






















— 1 
































i 

i 




























i 
i 


















! 
! 





































1 


i 


1 1 i 






i 


1 1 
1 J 






















1 



LEA REQUEST FORM (C) 

] 

i The LEA Request Fnrm focuses on an important facet of the ESCOE 

i 

= project: services to the LEAs. This form allows LEAs to request 

! services of ESCOE in the form of copies of printouts and matrices 

; (specified by Subgroup number), forms and materials, publications, 

i or direct services to the tl'A by the ESCOE. field staff. The latter 

i ^ 

! requests are written in the spaces at the bottom of the reverse side 

: of the form. The ESCOE staff made it a point to respond to such 

I requests immediately and to act on them as soon as it was feasible. 

i 
I 



\ i 
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BEHAVIORAL OBJECTIVE REPORTING FORM (#10) 
AFFECTIVE CAPABILITY 



This form is not to be filled out by individual teachers but coopera- 
tively by departments with participating LEAs. Pages 52-57 of 
Technical Report'Number 1 , on affective capabilities, should be read 
before attempting to use this form. 

Defining an affective capability as a positive or negative attitude 
or feeling toward an object, person, or idea, department members are 
to agree to at least five objects, persons, or ideas toward which 
their program is seeking to^ establish predetermined attitudes. By 
specifying a capability classification number within the Affective 
Capability Classification System, each department can express the 
extent or degree of feeling sought. The following are examples of 

1 items towards which attitudes might be sought: capatalism, democracy, 
tolerance of others, employers, self as a compatent person, dishonesty » 
becoming a productive citizen, vocational education. The affective 

j capability should be agreed to by all department members and sought 
for all students. It is assumed that these affective capabilities 
are ''end program" objectives* 

I This form was not extensively used by ESCOE= It is included here 
I in the hope that it may be useful to others intending to deal with 
I the affective domain in education. 

i 
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CRITERION-REFERENCED TESTS 



Background 

The 1968 Amendment to the Federal Vocational-Technical Education Act 
mandated the development of state-wide evaluation systems for the admin- 
istration and operation of federally supported vocational education. 
Parallel to this mandate the Research Coordinating Unit director for the 
Commonwealth of Massachusetts was in the process of completing some pre- 
design activities for the development of a vocational-technical education 
management information system. By 1969 the predesign of this system had 
moved intc the feasibility stages and specifications of the system were 
being developed. 

At this stage New York State, which already had a fine centralized 
testing program, became interested in the philosophy espoused by the Mas- 
sachusetts system and joined in the funding of a more intense feasibility 
test, which eventually became the source of the Performance Test Develop- 
ment Project. The Evaluation Service Center for Occupational Education 
(ESCOE) was funded in late 1970 and was housed in Amherst, Massachusetts, 
to test the feasibility of systems development based upon the principles 
of (1) local control and development of vocational curricula, (2) data- 
based feedback based up^n tailored performance tests, and (3) curriculum 
description through terminal behavior objectives. The following report 
deals with a subcomponent of the ESCOE system which was designed to de- 
velop perfomance tests as software support for the ESCOE prograiri. 
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Perfonmance testing is more a new reality as opposed to a new con- 
cept in educational testing. The concept grows out of the need felt by 
educators to sample actual performances of trainees as opposed to merely 
measuring s^/mptoms of desired (or intended) competencies through paper 
and pencil tests and then relying upon the predictive powers (i.e., pre- 
viously established associations of paper and pencil test scores to some 
hypothetical or observed criterion of competency in performance) of the 
test to infer competency acquisition. This felt need has grown in part 
from the inability of standardized achievement tests to deal with the 
unique objectives of a specific educational program, in part from the 
reportedly low correlations between measured skills and on-the-job (or 
in-the-shop) perfonnances , and in part from the lack of realism involved 
in the paper and pencil testing situation. 

Hence the performance test can be conceived of as a criterion-ref- 
erenced test, in that (1) it is objective or criterion-centered (in one- 
to-one correspondence with the extent component of a stated objective); 
(2) it seeks to ascertain a subject*s possession of a specific competency 
rather than to complete a comparison of the subject*s competency level to 
a previously measured norm group; and (3) it usually requires a dichoto- 
mous decision as to whether the competency has been demonstravad. The 
performance test can be construed to be a special case of the criterion- 
referenced test in that there is a definite attempt to establish fidelity 
between the sample observation of the performance test and the performance 
being sampled. 

In the evaluation of instructional programs in vocational-technical 
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education, the concept of performance testing is especially appropriate 
for several different reasons. First, performance tests can be hypothe- 
sized to produce more relevant and valid data concerning the instruction- 
al program output. Vocational program objectives tend to deal with compe- 
tencies which require concurrent behavior changes across several domains 
of instructional objectives. Hence the accomplishment of a vocational 
objective may depend upon the development of et psychomotor skill, the 
mastery of a cognate process, the acquiring of some fundamental facts, 
and the development of a particular attitude. Unlike paper and pencil 
tests, which emphasize the measurement of the cognitive aspects of the 
performance or observations which emphasize process and action components, 
performance tests possess the potential to measure the mixture of behavior 
domains appearing in the desired performance. The performance tost can 
therefore be argued to offer a valid means of measuring intended outcomes. 

Second, performance tests produce product records which can be stud- 
ied by teachers to diagnose the place in the instruction where a weakness 
may have occurred, aiding considerably their ability to tndAyze their 
instructional methods. Since the teacher can determine what aspects of 
the competency are missing, he can trace the point in his instruction 
where his objectives WGre not met. Also, since the product is concrete 
it can be kept longitudinally to analyze pupil growth at different stages 
of a multi-year program. 

Third, the nature of the data produced by performance testing con- 
tains the flexibility demanded by the information needs of an evaluation 
system. The tests are constructed in one-to-one correspondence to stated 
objectii^s,' thus enabling selection of test components from a data bank 
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situation in such a manner as to tailor the testing to the measurement 
of a un'^Que set of program objectives. Since the tests are objective, 
specific comparisons of small aspects of an instructional program are 
possible. Since the tests are criterion-referenced, skill attainment in 
£ particular area of interest can be ascertained; hence output of instruc- 
tional programs can be described relative to percentage of skill develop- 
ment . 

Restraints on Test Development 

The design of the performance tests had to take into account both 
the philsophical and the operational structure of ESCOE. At times both 
of these structures served as restraining and occasionally frustrating 
hurdles for the test development team. 

The p^rrtosophical nature of ESCOE provided the foundation of princi- 
ples which are believed to have caused the performance tests to be unique. 
Since the objectives were generated by each local school, several very 
similar- objectives appeared for a single behavior within a subject. Dr. 
D':ivid S^jrliner, now with the Far West Laboratory for Educational Research 
and Development, invented a process to state these similar objectives into 
a synthesized form accompanied by item changes providing for the unique 
characteristics of each objective. Thus, if enough objectives from dif- 
feretrt schools were collected to represent the curricula, by synthesizing 
those objectives one could arrive at a statement of all desirable behav- 
iors within one curriculum. 

The raw objectives based upon the curricula of each of the partici- 
pating schools were synthesized to identify the major behaviors within a 
curriculum area. Hence, if the process worked ideally within a curricu- 



lum area a linear set of behaviors was produced. The degree to which this 
process failed to produce such a linear array of behaviors constituted the 
first major restraint. If a singular listing of behaviors could not be 
\ gained, then singular test items could not be written. 

A second philosophical principle which developed into a restraining 
factor was the decision to test only locally-maintained objectives within 
a specific program. This principle actually involved several implications 
for testing. First, a student would be tested only on the objectives main- 
tained by the curriculum he was receiving. Therefore, the test items had 
to be described in a form indicating one-to-one correspondence with the 
synthesized objectives so that the local teacher could select only those 
items maintained for his course. This selection pattern, however, did 
increase the logical assumption that the tests possessed high validity 
in regard to the courses for which they were designed to measure outcomes. 
Second, each item had to be indeperdent in its ability to be administered, 
since previous or adjoining items would not necessarily be administered 
with it. This item independence served as a restraint to test development 
in that objectives could not be clustered into tasks involving several 
test items. 

The third restraint involved both philosophical and operational as- 
pects in that two forms of scoring were preferred by the two cooperating 
states. Philosophically, the state coordinators differed on the location 
of scoring; this disagreement became a restraint to test development in 
that the items developed had to be scorable both in the local school and 
at a central test center. Three forms of scoring meeting this restraint 
were adopted, with choice of scoring form depending upon the nature of 
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the individual item. Two of the forms are based upon meeting the restraint 
with a single scoring process. The third form requires two different pro- 
cesses in order to meet the dual scoring restraint. 

The scoring approaches requiring only one process are (1) the caliper 
or mechanically scored form and (2) the selection of correct response form. 
In the mechanically scored approach, several measured settings can be 
placed in a test scoring kit; the student or teacher records by label the 
setting which fits the final product, A key of correct setting labels can 
then be referred to, producing a dichotomous score for the product in terms 
of size tolerances. In the selection of correct response approach, correc- 
tion keys can be applied directly to the student^s response. In both cases 
either a central office or an individual classroom teacher can use the keys. 

The third scoring form is not as simple, since two types of scores are 
required to meet the dual-use restraint. This scoring form is necessitated 
by the many tasks in the vocational curriculum which require expert observ- 
er judgment for the determination of performance quality. The two types of 
scoring needed for these items are (1) structured criteria for observation, 
and (2) pictorial records (color-coded to facilitate central scoring). The 
structured criteria for observation communicate to the individual teacher 
what aspects of the product to check in order to judge the performance suc- 
cessful. These criteria would be used in class. In the pictorial scoring 
process, camera angles have been described which would allow Polaroid pic- 
tures to be taken as records of the finished product. Color-coding the 
criteria checks would enable observers in a central location -co determine 
the quality of the performance. 

Each of these three approaches provides a means through which credible 
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and unbiased scores can be obtained. All of the processes can be scored 
by individual teachers and used within a classroom setting without the aid 
of a central scoring station. The fourth restraint to test development 
arises at this point, since it is impossible to arrive at an immediately 
usable set of norms through the current scoring system and the dichotomous 
item response without implementation of a program designed to gather enough 
data to norm the tests. 

Two other restraints were present throughout the test development proj- 
ect, both operational in nature. First was the quality and quantity of the 
behavioral objectives themselves. Few if any of the curriculum areas were 
fully described, and the tests developed are limited to described curricu- 
lum. In two test areas, more items were developed and the synthesization 
process was repeated in order to sharpen the synthesized objectives. In 
these cases much curriculum had been left undescribed and the fill-in pro- 
cess aided considerably in explaining the descriptions. However, complete 
and multiple sets of items were not available from each school; therefore 
the test items may be lacking in content validity in cases of consultant- 
written items, may be representative of several behaviors, and may hence 
be difficult to test or represent only a small segment of the previously 
unwritten curriculum. 

The second operational restraint was that of time. Although the budg- 
et was small, the seriously close deadlines in development work made time 
an even greater restraint. Creativity is sometimes especially evasive 
under the pressure of deadlines and within the constraints of administra- 
tive conflict. Still, the time dimensions were met in terms of design. 
Since schools were closed during the critical month of June, illustrations 
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of sofTie items of the tests could not be produced ^ therefore only plans, 
item descriptions, materials descriptions and admini ration instructions 
could be developed. 

A final restraint can be observed in the language in which the pro- 
posal was written. First, several terms apparently changed in meaning or 
in relevance to the project once development began. One apparent change 
occurred in the description of sixteen tests for four areas. One test 
for each level of a curriculum area cannot be developed so as to be 
equally relevant to all schools. Since the schools maintain different 
objectives, different items must be assigned to each school, even on the 
same level. Hence a more appropriate process becomes the development of 
an item bank from which tailored tests can be developed for each individ- 
ual program. Second, the time restraints and the differences in the nature 
of curriculum required different kinds of tryouts, making the language of 
the proposal seem sometimes inappropriate. 
Purposes of the Test Development Project 

The design of the test development project included not only the 
goal of producing tests as products but also the goal of tastabli shing 
feasibility of the test development effort across a broad spectrum of 
vocational-occupational curricula. For this reason four different areas 
of vocational curricula were selected for test development. These four 
areas differed in hypothetical difficulty of test development. The areas 
chosen were machine shop, woodworking and carpentry, electronics, and au- 
tomobile mechanics. The automobile mechanics area was hypothesized to be 
the most difficult since manufacturers determined the curriculum, which 
therefore differed across competing manufacturers. ♦ 
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The performance tests were hypothesized to be sufficiently flexible 
to fulfill many purposes of a comprehensive evaluation system. Because 
of their proximity to the desired outcomes, performance tests were hypoth- 
esized to serve as (1) student diagnostic and prerequisite instruments > 
(2) diagnostic instruments for the analysis of instruction, (3) criterion 
instrunents, (4) measures of classroom achievement, and (5) program suc- 
cess indicators. Each of these uses has already been piloted to some 
extent. 

The performance tests as developed have several application conven- 
iences. First, since the test items are paralleled to synthesized objec- 
tives, computer selection of test items or "synob" comparison of items 
can be used as a methodology for tailoring tests to instruction. Second, 
since the conceptual frames of the tests can be described, each test has 
built-in potential updating and extension by the classroom teacher. 
Problems Encountered 

Problems occurred from three viewpoints* First was the problem of 
lack of known direction, a handicap which often occurs in the area of 
development. Second was the problem of lack of perfection or completion 
of the objectives used as raw materials for the development of test items. 
Third was the problem of contending with dual scoring requirements and 
with several different kinds of program emphasis and structure. 

The first problem has been emphasized recently with the development 
work done on criterion-referenced testing. From a conceptual point of 
view, the criteria previously used to determine the quality of norm-ref- 
erenced tests can no longer be used for criterion-referenced tests. Since 
the measurement strategy of the criterion-referenced test and the perfor- 
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mance test is to determine the possession of either a skill or the capa- 
bility to carry out an activity or process, the degree to which the test 
differentiates between subjects taking the test does nothing to indicate 
test quality. Unlike the norm-referenced test, in which measurement 
strategy is to distinguish between subjects, the performance test cannot 
be hypothesized to produce large differences across subjects nor can any 
specific level of difficulty be expected. Hence, average levels of dif- 
ficulty and large differences between subjects do not indicate quality 
of the performance test. 

In performance testing, some concepts of reliability still appear 
useful, while others appear to have lost their relevance. Reliability 
over time, or test-retest reliability, is still meaningful as long as the 
time between tests did not include opportunity for the subject to acquire 
the skill in question. Since performance tests are designed so that each 
item does not necessarily refer to the same skill or activity, reliabil- 
ity indices dealing with homogeneity of the test no longer appear to be 
relevant criteria for test quality. 

The degree to which the items of a performance test cover the skills 
of an area and approximate actual performances required operates in a 
similar relationship to the performance test as that of a prediction Index 
to a norm-referenced test. This degree of similarity might be compared to 
the concept of fidelity so often used in the recording industry. 

The second problem involved the quality of the raw materials used. 
As should be expected, the synthesis process does not apply evenly to all 
areas and was not applied with the same consistency to each set of objec- 
tives. In the machine shop curriculum area, between 70 and 80 percent of 
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the content was described by the objectives. These objectives possessed 
adequate depth across skill areas to enable the synthesis process to pro- 
duce clear synthesized objectives describing unique performances. The 
creation of items parallel to t'le :;ynthesized objectives and possessing 
the independence and flexibility required by the philosophy of the system 
was a straightforward process. 

In the woodworking area, between 60 and 75 percent of the content was 
described. Unfortunately, the synthesizers of the raw objectives failed 
zo produce synthesized objectives which dealt only with single perfor- 
mances. Instead, the raw objectives were synthesized by similar or relat- 
ed behaviors and the product of this process was a matrix of similar per- 
formances (rather than a single performance) with several form changes 
denoting differences in conditions and extents across schools. Since 
these products seemed usable, the decision was made to produce a matrix 
of test items generated in one-to-one correspondence to the performances 
included in each synthesized objective. This decision was the source of 
some time lost due to the expanded number of test items which had to be 
written; however, this increase in items was accompanied by a large in- 
crease in test specificity, which in turn increases the degree to which 
the performance test can be tailored to fit a given instructional program 
without any noticeable loss of efficiency of the item banking process. 

Due to the variance of material and the limited scope of the objec- 
tives developed for the electronics curriculum area, a decision was made 
to rewrite many of the synthesized objectives. For more than one-half of 
the contract period two of the test development team members struggled to 
find a format within which the scope of the electronics curriculum could 
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be described. By expanding the number of conditions it was found that 
classes of performance could be described by synthesized objectives. 
Hence, through considerable rede::vign and a small set of compromises of 
the synthesis process involving uniqueness of performances and allowance 
of performance form changes, subcol lections of electronics objectives 
could be written which would allow test development along similar con- 
ceptual lines as those followed in the development of the machine shop 
test. Results of the test development effort again produced item banks, 
as in the two previous test areas, with the Items possessing similar re- 
lationships to the synthesized objectives. 

In the area of automobile mechanics, less than 50 percent of the 
content was described by the raw objectives. Many of the subdivisions 
of content were too. sparse to allow for the development of synthesized 
objectives. In addition^ the synthesis process applied seemed irregular 
across blocks and units. The level of abstraction of behaviors described 
by the raw objectives and the interdependence of the performances raise 
questions concerning the appropriateness of the synthesis process in this 
area. Certainly, the limited number of usable synthesized objectives and 
the necessary revisions of the existing objectives made the decision to 
rewrite the objectives essential. Revision of the curriculum descriptions 
were made in relationship to the job orientation of the curriculum. Test 
items were written around standard mechanics tasks as described in the 
automobile mechanics curriculum. In some of these items, synthesized 
objectives are tested in a format which includes a cluster of the objec- 
tives provided by the ESCOE system. In other items, only parts of ESCOE- 
produced objectives are included in the new synthesized objectives being 
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tested. Once a test item has been constructed, the process can be reversed 
so that system capability as achieved in the other three test areas can be 
gained. Because of their time-consuming nature, tasks in the curriculum 
such as disassembly and reassembly of a motor or transmission were not in- 
cluded as individual test items. Instead, either sample tasks from the 
large unmanageable task or written or pictorial selection items were 
created to test these phases of the curriculum. 

The third problem area encountered was the difficulty involved in the 
existence of two separate scoring requirements and in the time limitations 
of the test development project. It was not always possible to produce 
useful in-class scoring of the performance item and credible, objective 
centralized scoring of the performance item through application of the 
same scoring process. Therefore some items are suspected to produce more 
useful scores in the classroom than in a central scoring situation, while 
the reverse is suspected of other items. Only time and study of the tests 
can alter or affirm these suspicions. It is unfortunate that systematic 
refinement of the woodworking, electronics, and automobile mechanics tests 
is not planned to occur along the same lines as those applied to the ma- 
chine shop test. ' 

Development and field test procedures, item bank descriptions, rec- 
ommended analysis procedures, and uses for the four tests are contained 
in the individual test development reports. 

The four tests which have been developed and are appended as separate 
volumes to this report are the following: 

Performance Test -^or Auto Mechanics 
by Jim C. Fortune 

Center for Educational Research, School of Education 
University of Massachusetts, June 1972 



-97- 



Cri ten' on- Referenced Item Banking in Electronics 
by Willi am Phillip Gorth and Hariharan Swaminathan 
Center for Educational Research, School of Education 
University of Massachusetts, June 1, 1972 

Performance Test Development in Machine Shop 
by Jim C. ForC"une 

Center for Educational Research, School of Education 
University of Massachusetts, June 1972 

Woodworking Objective and Test Item Bank 

by Ronald K. Hambleton, Center for Educational Research 

School of Education, University of Massachusetts 

and Francis Olszewski, Smith Vocational High School 

Northampton, Massachusetts. 



Copies of the performance tests will be sent with this report as 
separate volumes* 



-98- 



VII 
APPENDICES 



ERIC 



APPENDIX A 



BLOCKS AND UNITS 



Field of Study 
Major Group 



For Example: Technical 



Engineering Technology 
Civil Technology 

Elementary Surveying 
Tap I ng 

Correction for long or 
short tape 



Subgroup 
Block 
Major Topic 
Unit 



'7he Blocks and Units serve as categories wHhin a subgroup by which 
objectives are classified and coded so that they may be stored and 
retrieved systematically. Each breakdown list is the result of at 
least two instructors from different LEAs working together in identi 
fying the Blocks and Units of instruction for a particular program. 
It should be made clear that this breakdown is an arbitrary list to 
be used for the ESCOE data system. There is no suggestion that this 
is the 'correct' breakdown for any program. The Block and Unit 
Breakdowns provide a common language through which LEAs may share 
behavioral objectives and still maintain in their o n LEA different 
descriptions, if they so desire. 

"The Block and Unit lists are always open-ended. If an objective 
cannot be fitted into one of the existing Blocks and/or Units, 
another term describing the stated performance may be used." 



Because of the differing depth and comprehensiveness in secondary 
and post-secondary programs, a Block in a post-secondary curriculum 
description is much broader than a Block at the secondary level: it 
is essentially equivalent to a semester course. Specificity in a 
post-secondary program equivalent to specificity in a Block at the 
secondary level is to be found in the Major Topic. The Major Tonic 
category is included in the example given here; the objective to 
which this breakdown applies was written for a post-secondary area. 



from Memo to ESCOE Facilitators, June 6, 1972 
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Subgroups for Which Blocks and Units 



SECONDARY 



USOE 




CODE 


SUBGROUP 


U /UoUc 


rr dct icdi iNUrsing 


1 /03L) 1 


Auto DOdy & Fender 


1 70302 


Automobile Mecnamcs 


170701 


Interior Decorating 


170702 


Window Display 


170703 


Product Design 


170799 


Other Cominercial 




Art Occupations 


171001 


Carpentry 


173601 


Millwork and Cabinet 




Making 


173699 


Other Woodworking 




Occupations 


171300 


Drafting Occupations 


1 "71 A nil 

1 71401 


Indus tri al El ectri ci an 


1 /lOOZ 


Electricity 


1 71502 


Electronics 


171901 


Composition, Make-up, 




Typeset 


171902 


Printing Press 




Occupations 


171903 


Li thography 


171905 


Silk Screening 


171999 


Other Graphic Arts 




Occupations 


172302 


Machine Shop 


172305 


Sheet Metal 


172306 


Welding and Cutting 


172397 


Precision Sheet Metal 


172398 


Metal Fabrication 



Have Been Developed ^ 

POST-SECONDARY 
USOE 

CODE SUBGROUP 

030101 Principles of 
Accounting 

070301 Associate Degree 
Nursing 

160106 Civil Technology 

160113 Mechanical Technology 



SECONDARY (continued) 



USOE 




CODE 


SUBGROUP 


172399 


Other Metal 




Fabrication 


172901 


Baking 


172902 


Chef/Cook 


172903 


Meat Cutter 


172999 


Other Meat Cutter 




Occupations 


172904 


Waiter/Waitress 


172602 


Cosmetology 


171007 


Pipefitting 


1 71 007 


Plumbing 


172301 


Foundry 
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APPENDIX B 



DATA STORAGE SYSTEM 



The storing, sorting, and listing of the raw and synthesized objec- 
tives was done primarily at the University Massachusetts in 
Amherst on its CDC3500 and CDC3800 computers. All programs were 
initially written in FORTRAN. In the fall of 1971, the raw objec- 
tive data bank was transferred to the Massachusetts Department of 
Education, Research and Development Center. Some COBOL programs 
have been written there. 

To make up the data bank of raw objectives, approximately 10,300 
objectives were written onto data forms by the facilitators. Cards 
were keypunched from these forms, read into the computer, and stored 
in variable length records on magnetic tape. Each objective on the 
tape includes an identification number and codes as to school and 
subject area, as well as the written objective itself. Also included 
in the objective records are code numbers for the related subject 
areas, the number of students taking a particular objective, and the 
level within the school program where the objective is first taught. 

The tape of raw objectives was sorted and major computer printouts 
in order of raw objective I.D. number, subject area (field, group, 
subgroup, block, and unit) and school were provided. A program was 
also written to make up a matrix by school and subgroup telling how 
many objectives in each block and unit were received from a particu- 
lar school. Other minor programs for finding errors in the data, for 
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DATA STOR^^^^ SYSTEM (continued) 



getting simple statistical information from the bank of objectives, 
and for listing only spec^ ^ic sections of the bank of objectives were 
written and run. 

Four subgroups of the raw objectives amounting to 2571 objectives 
have been synthesized. The resulting 343* synthesized objectives 
were punched onto cards, read into the computer, and stored on mag- 
netic tape. A computer program for listing this tape in the desired 
format and one which will make up a matrix of the synthesized objec- 
tive I.D. numbers in each su')group, block, and unit have been written 
and run. 

f 

In the future both states will continue to use the banks of raw and 
synthesized objectives. A set of punched cards for each state is 
being prepared, and Massachusetts and New York will, therefore, ini- 
tially have identical copies of the entire data bank. The data banks 
will be in use in New York at the Hudson Valley ESCOE, Hudson Valley 
Community College, in Troy, and in Massachusetts at the MISOE project 
Division of Occupational Education, Massachusetts Department of Edu- 
cation in Winchester. 



* as of August 1971 
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DATA STORAGE SYSTEM: CODING HEADER 



The function of the coding header accompanying every objective 
submitted to ESCOE —is to provide code numbers for the vav^ious 
dimensions by which objectives may be retrieved from the data bank. 

Pont, n2B ^^^^^^y ''572 

EVALUATION SERVICE CENTER FOR OCCUPATIONAL EDUCATION 
Behavioral Objective Reporting Fonn 



ID Number 



Yr . T N C , St. > City-Schoo1 i Lvt i No. Tk. 



c, 


ip. 


CI 








G 


B 






N 




Related S 


ub.^, U 


i$c 
















F1 1 G 


1^ 






t 














1 











LEGEND : 

ID Number - number assigned to every objective by ESCOE staff, in 
the order in which it is received at ESCOE; e.g., the 
sixty-seventh objective received is coded as #000067. 

Year - the year in which the objective was written 

TNC - [for keypunching usej 

St. - state, i.e.. New York or Massachusetts, in code 

City-School - USOE code number (6 digits) assigned to each LEA 

Level - a two-digit code number identifying program length and 

level and the'year of study in which the objective is 
offered 

No.fk. - number taking. I.e., the number of students to which this 
objective is taught in a school year 

Cap. CI. - capability classification, i.e., psychomotor, cognitive, or 
affective, in code 

Fl - Field 

GR - Major Group 

SG - Subgroup 

BL - Block 

MT- Major Topic 

UN - Unit 
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THE PROCESS OF RECEIVING AND STORING OBJECTIVES AT ESCOE 



1. Objectives received at ESCOE (by mail or hand-carried) 

2. secretary scans objectives: 

a. if adequate (legible, complete, etc.) step 3 is initiated 

b. if inadequate, objectives referred to editorial staff who 
either return the objectives to the LEA with comments or 
make minor adjustments and send to step 3 

3. secretary: 

a. stamps date on first and last objectives of each subgroup 
received 

b. stamps ID number on each objective 

c. enters data in logbook (number of objectives in a subgroup 
received, subgroup code number, ID number of first and last 
objectives in subgroup, date, etc.) 

4. objectives labeled "to be coded'* and filed by ID nunter within 
subgroup and LEA 

5. staff: 

a. codes objectives 

b. logs out objectives to be keypunched 

c. labels objectives "to be keypunched" and files 

6. objectives taken to be keypunched 

7. objectives returned after punching and: 

a. logged in by person returning them 

b. labeled "have been keypunched" and filed. 
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APPENDIX C 
ABSTRACTS OF ESCOE PUBLICATIONS 



Technical Report Number 1 (March 1971) - A brief description of the 
Project, a coding and reporting guide manual, a performance 
objective guide, and a related bibliography. 



Instruction Manual: Synthesized Objective Package - A prototype 
training package guide for the development and use of synthesized 
objectives; complete information on the feedback loop between the 
LEAs and ESCOE. October 1971. 



Behavioral Objective Training Package - A complete introduction to 
ESCOE and behavioral objectives, including the process of sub- 
mitting objectives to ESCOE; for the use of facilitators and 
teachers affiliated with ESCOE. November 1971. 

Programmed Text - A self-instructual guide for learning to write 
and recognize well -written behavior*^ objectives. 
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APPENDIX D 



GLOSSARY 



Affective Domain - The sphere of learning that deals with feelings 
or attitudes; one of the Capability Classifications. 

Behavioral Objective - A statement describing in observable terms 

what a student will be able to do after successfully complet 
ing a prescribed unit of instruction. 

Block - Largest instructional segment within a Subgroup. 

Capability Classification - A system for coding the abilities 

demonstrated in the performance of a behavioral objective, 
namely, psychomotor, cognitive, or affective capabilities, 
or combinations thereof; domain. 

Category Breakdown - The terms ESCOE uses to categorize occupational 
programs of study: Field of Study, Major Group, Subgroup, 
Block, Major Topic, Unit. 

Coding Header - The numerical tabulation of data on Behavioral 
Objective Reporting Form (#12B), by which an objective 
is classified and identified, as, for instance, the LEA 
writing the objective, the year in which it was written, 
the Block and Unit, etc. 

Cognitive Domain - The sphere of learning that deals with mental 
and intellectual capabilities. 

Conditions - The exact circumstances, e.g., materials and 

procedures (or restrictions on them), under which an 
objective is performed; one of the three components of 
a behavioral objective (see also Extent, Performance). 

Criterion-Referenced Test - The evaluation instrument used to 

assess the degree to which an individual meets predeter- 
mined standards on specific criteria statements (behavioral 
objectives). 

Domain - An educational sphere of learning, i,e.. Affective, 
Cognitive, Psychomotor. 

ESCOE - Evaluation Service Center for Occupational Education. 

Extent - The criteria--qual ity , tolerances, accuracy, percentage 
of correct answers, etc. --used to measure the performance; 
one of the three components of a behavioral objective (see 
also Conditions, Performance). 

Facilitator - ESCOE liaison person in a local educational agency. 

Family - A collection of allied Subgroups making up a Trade. 
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GLOSSARY (continued) 



Feedback - The process of communicating the products of the system 
to the users and by which the users react to the products. 

Field of Study - Broadest category of occupational classification, 
e.g., Trade & Industry, Health Occupations. (See Category 
Breakdown) 

Fixed Text - That portion of a synthesized objective that is not 
optionable, or does not vary. 

Form Changes - The options available within a synthesized objective; 
the variable text. 

LEA - Local Education Agency - A school or group of schools affiliated 
with ESCOE that operates under one administrative organization, 
such as a high school , trade school, vocational-technical 
school, BOCES center, community college, skills center, etc. 

Major Group - Classification of occupational programs within a 
"Field of Study"; e.g., with Health Occupations: Dental 
Services, Medical Services, Nursing, etc. (See Category 
Breakdown) 

Major Topic - A division with a Block, created for the particular 
characteristics of post-secondary programs. (See Category 
Breakdown). 

Norm-referenced standards - A testing approach in which students' 
performai'^ces are compared to the performances of others 
in the same reference group, such as achievement tests 
and aptitude tests. These tests provide no direct indica- 
tion of the individual's degree of proficiency in relation 
to specific criteria. 

Performance - The exact observable behavior a student who has 

mastered the requirements of an objective will manifest; 
one of the three components of a behavioral objective (see 
also Conditions, Extent). 

Printout - Printed computer output. 

Psychomotor Domain - The sphere of learning that deals with physical 

skills which require muscular coordination and varying degrees 
of strength* 

RAWOB - Raw Objective - A behavioral objective, before synthesis. 

Subgroup - Occupational programs within a Major Group classification. 
(See Category Breakdown). 
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GLOSSARY (continued) 



SYNOB - Synthesized Objective - Raw objectives similar in performance 
combined into one statement 

Unit - Instructional segment within a Block or Major Topic by which 
an objective is classified (see Category Breakdown). 

U.S.O.E. - United States Office of Education; ESCOE used U.S.O.E. 

code numbers for Field of Study, Major Group, and Subgroup 

to identify trades, academic subjects, occupational areas, etc. 

Variable Text - That portion of a synthesized objective which is 
optional e; the form changes. 
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APPENDIX F 



AVAILABILITY OF ESCOE PRODUCTS 



Source information for the raw objective printouts and synthesized objecr 

tive printouts as well as A ProgramTOd Text for Writing Behavioral Ob,1ectives 

will be available in Massachusetts from: 

Mr. Ronald Saris, Director 
Research Coordinating Unit 
Division of Occupational Education 
State Department of Education 
182 Tremont Street 
Boston, Massachusetts 02111 

and in New York from: 

Mr. Walter Long, Director 
ESCOE 

Hudson Valley Community College 
80 Vandenbera Avenue 
Troy, New York 12180. 



The final report and all separately bound appendices will be available 
through the ERIC System. 



For information about the series of overhead transparencies developed by 
ESCOE, write to Mr. Long at the above address. 
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